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Strategies. The study focuses on changes in program . administration 4 years 
after the 1994 enactment of Goals 2000 and ESEA. Specifically, the study asks 
how state program managers are, implementing the laws’ provisions, how 
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predecessor programs, and what federal and state factors have influenced 
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EXECUTIVE SUMMARY 



Study Purposes and Design 

The Goals 2000: Educate America Act and the Improving America’s Schools Act (IASA), 
which amended the Elementary and Secondary Education Act (ESEA), were enacted in 1994. 

Together they were designed to provide a comprehensive system of support for state and local 
education reform initiatives that would enhance children’s educational achievement. Specifically, the 
laws allowed state administrators of federal programs to coordinate and consolidate their administrative 
functions so as to minimize the burden and cost and thereby redirect their programs to support broader 
state policy initiatives, such as the implementation of standards. These laws also offered greater 
decision-making authority and flexibility to local administrators and teachers in exchange for greater 
responsibility for student performance. 

This study, conducted under contract with the Planning and Evaluation Service in the Office of 
the Under Secretary, U.S. Department of Education (ED), focuses on the work of state administrators 
of federal programs. It follows up on baseline information collected during late fall 1996 and early 
winter 1997, analyzing the ways in which state administrators have continued to respond to the new 
laws. Data for the follow-up study were collected in summer and fall 1998. The programs included in 
the follow-up study are: the Goals 2000: Educate America Act; Title I-A: Improving Basic Programs 
Implemented by Local Educational Agencies; Title I-B: Even Start Family Literacy; Title I-C. 

Education of Migratory Children; Title I-D: Prevention and Intervention Programs for Children and 
Youth Who Are Neglected, Delinquent, or At-Risk of Dropping Out; Title H: Eisenhower Professional 
Development Program; Title III, Subpart 2: Technology Literacy Challenge Fund (TLCF); Title IV: 
Safe and Drug-Free Schools and Communities; and Title VI: Innovative Education Program Strategies. 

• This study focuses on changes in program administration four years after the 1994 
enactment of Goals 2000 and ESEA. Specifically, the study asks: (1) how state 
program managers are implementing the laws’ provisions; (2) how implementation has 
changed when compared with state practices under the predecessor programs; and (3) 
what federal and state factors have influenced these changes. The study explores the 
extent to which managers administered federal programs in ways that: (1) make use of 
increased flexibility across programs; (2) make programs more accountable for student 
performance; and (3) support improvements in teaching and learning. 

• The follow-up study differs from the baseline study in that it: (1) did not include the 
Education for Homeless Children and Youth Program because that program is being 
evaluated under a separate ED-funded study; (2) included site visits to eight rather than 
13 states and interviews with program administrators at both the state and district levels 
(two districts per state); (3) had a special focus on the implementation of the Education 



Flexibility Partnership Demonstration Program (Ed-Flex); (4) included Title III, the 
Technology Literacy Challenge Fund; and (5) looked at the extent to which states are 
collecting data that would inform the program performance indicators developed by ED 
under the Government Performance and Results Act. 

Surveys were administered during late summer and fall 1998 (approximately four years 
after the reauthorization of ESEA, four-and-a-half years after the authorization of Goals 
2000, and two years after the baseline study data were collected) in all 51 state 
education agencies (including the District of Columbia) to state-level managers of each 
of nine federal programs, plus administrators knowledgeable about Ed-Flex, for a total 
of 468 possible respondents. Each follow-up survey was administered by telephone as a 
personal interview with standard questions. Out of a possible 468 surveys, 447 were 
completed, a response rate of 96 percent. 



Flexibility: Do States and Districts Continue to Experience New Latitude in 
Implementing the Law? 

Two-thirds of state administrators of ESEA programs surveyed in 1998 believed that their own 
flexibility had increased in the four years after reauthorization, whereas in 1996-97, Title I was the 
only program in which a majority of administrators reported an increase in administrative flexibility. 
When asked whether they need additional flexibility, most administrators reported that the legislation 
provided them with sufficient flexibility. 



This study looked at the ways in which state program administrators have continued to make use 
of the flexibility provisions in the new laws and the extent to which they have moved beyond attention 
to procedural provisions— such as consolidated planning and cross-program communication— and begun 
focusing on the task of using the flexibility provisions to align program services and operations to 
support state content and performance standards and thereby improve student achievement. 

• In describing the administrative flexibility available to them since reauthorization, state 
program managers expressed two major themes: (1) the opportunity to coordinate and 
collaborate with other federally funded programs, and (2) the opportunity to support 
local reform plans. The first theme was also prominent in responses collected during 
the baseline study; the second, however, grew in frequency, suggesting that state 
administrators may have moved further along in their thinking about the ultimate 
purpose of their programs: to support local reform efforts. Indeed, many program 
managers described efforts to help districts find ways— sometimes creative ones— to use 
federal program resources to meet local needs rather than using administrative 
flexibility to simply administer programs within an explicit framework of program 
regulations and requirements. Few administrators, however, mentioned the 
relationship between flexibility and improvements in program services or student 
outcomes. 
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• Some program administrators explained that although they may have the flexibility they 
need from the federal level to successfully administer their programs, state-level 
policies and practices can curtail the flexibility that the reauthorized ESEA affords 
them. 

• Several program managers expressed frustration at not being afforded the same 
flexibility in responding to federal reporting requirements that they enjoy with respect 
to program planning and funds consolidation; some argued that ED’s program-specific 
reporting requirements work to dissuade program managers from coordinating program 
services and activities as well as causing states to impose undue reporting burdens on 
districts. 



State-Level Program Coordination 

In 1996-97, we learned that almost all state administrators of the programs included in the 
follow-up study participated in the development of their state’s consolidated plan. What we did not 
know was to what extent consolidated planning was translating into coordinated program administration 
and operations. Moreover, we did not know whether and to what extent administrators were 
organizing program administration and operations around supporting state goals or whether 
consolidating administrative funds was translating into increased administrative and operational capacity 
among programs. 

• Most state administrators (81 percent) reported conducting specific administrative or 
operational activities in coordination with other federally funded education programs, 
including: (1) providing technical assistance to districts and schools; (2) holding local 
application and planning workshops; (3) monitoring local projects; and (4) making 
decisions with respect to allocating program resources to districts and schools. Despite 
these efforts, state administrators have only just begun coordinating certain 
administrative and operational activities, and only with a core group of programs 
(typically, Titles I, VI, Eisenhower Professional Development, and Safe and Drug-Free 
Schools and Communities). 

• Roughly one-fourth of the administrators for the smaller programs— Migrant Education, 
Even Start, and Neglected or Delinquent— said that they were not coordinating any 
administrative or operational activities with other federally funded education programs. 

Many managers of the formula-based programs explained that the timelines and 
deadlines for applications for most discretionary grants programs simply do not match 
those for formula-based programs, making coordination difficult. 

Consolidated administrative funding . The IASA law allowed states to make a change in the 
way they accounted for state-level program administration funds: it authorized them to consolidate into 
a single pool the administrative set-asides under Title I, Even Start, Migrant Education, Neglected or 
Delinquent, Eisenhower Professional Development, TLCF, Safe and Drug-Free Schools and 



Communities, Title VI, and Goals 2000. The notion behind this additional flexibility was that it would 
“make it easier [for state administrators] to plan across programs” (U.S. Department of Education, 
1996a; p. 7). 

• Fewer states (18 in 1998 compared to 28 in 1996-97) reported that they consolidated 
administrative funds. The decline in the number of states consolidating funds may be 
explained, in part, by the significant turnover many SEAs experienced in the past few 
years; consequently, some program managers are fairly new to their positions and may 
simply not know whether their program’s administrative funds are part of a 
consolidation. 

• Looking at all the state administrators across all states who said their agency had 
consolidated administrative funds, 69 percent reported some effect on their own work, 
with no significant variation by program. The tone of most comments from state 
administrators in 1998 regarding the effects of funds consolidation, however, was 
decidedly negative. Some complained that consolidation had resulted in their having 
less money to administer their programs and to award to subgrantees. Other 
administrators complained that they had lost control of their program resources as a 
result of consolidation. 

State participation in the Education Flexibility Partnership Demonstration Program or 
“Ed-Flex. ” Established by the Goals 2000: Educate America Act, Ed-Flex gives participating states 
the “power to waive requirements of certain federal education programs, including Title I and the 
Eisenhower Professional Development Programs” in exchange for increased accountability for results 
(Ed-Flex Fact Sheet, ED, 1996). 



Among the 12 states that received Ed-Flex authority, administrators reported that the 
local waiver provision was, at best, underused and at worst, ignored. Of the 10 
administrators who were able to estimate the number of districts in their states that had 
received waivers, seven said the number was 20 or fewer. The most common 
explanation administrators offered for underuse of the waiver authority was that the 
reauthorized legislation grants programs sufficient flexibility, thereby rendering Ed- 
Flex superfluous; as one administrator pointed out: “There is so much flexibility built 
into ESEA that there is not a big need for more. ” 

Most administrators in the 12 states that received Ed-Flex authority reported including 
student performance among their criteria for assessing the impact of waivers and said 
they will revoke district waivers as a result of poor student performance— although 
none had, as yet, done so. 

Despite their rather limited use among districts, few state administrators would be 
willing to discard waivers as meaningless: most (8) say they believe— to some or to a 
great extent— that the Ed-Flex waiver authority (1) furthers state reform efforts, and (2) 
improves the coordination of federal resources with state reform efforts. 
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Local Administrative Flexibility 



The Goals 2000 and IASA amendments aimed to increase flexibility in schools and school 
districts so that administrative requirements would not impede progress in helping all students meet 
high standards. The survey asked state administrators to report on the extent to which the 
administrative flexibility afforded state education agencies was being extended to the local level. 

• About 57 percent of all program administrators reported that they required or accepted 
consolidated applications from districts. Although this shows a slight decrease from 
1996_97 (j.e., 60 percent of program administrators required or accepted consolidated 
applications), follow-up survey data showed an increase in the percentage of 
administrators who reported requiring— versus merely accepting— consolidated local 
plans or applications (30 percent in 1998 versus 22 percent in 1996-97. 

• State administrators of discretionary grants programs more frequently reported 
requiring separate subgrant applications than did administrators of formula grants 
programs. Administrators who continue to require separate local applications explain 
that discretionary grants programs operate on different grant cycles from the formula- 
based programs. 

• Interviews in 1998 with local administrators suggest that states were not making it easy 
for districts to submit consolidated plans. According to local administrators, state 
managers do not know what a local consolidated plan ought to look like— nor how they 
can support local program coordination — and therefore are unable to offer districts 
much assistance. 



Accountability: To What Extent Are States Implementing Standards-Based 
Accountability Systems? 

The follow-up survey explored the extent to which, four years after reauthorization, state 
administrators were organizing their work around aligning program services and operations with state 
content and student performance standards in an effort to improve student achievement. In addition, it 
explored the extent to which state administrators were using a variety of accountability tools— including 
student performance data, program implementation data, and program monitoring systems— to press 
vigorously for improved student performance. 



Content Standards; How Are They Affecting State Administration of Federally Funded 
Programs? 

Four years after implementing the law, it appears that state program administrators have come 
to recognize that supporting standards-based reform is part of their programs’ purpose. Moreover, 
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many administrators in their responses to open-ended survey questions mentioned raising student 
achievement the ultimate goal of standards-based reform— as among their more fundamental program 
goals. 



Few respondents (6 percent) said that standards are “not relevant” to the services their 
program provides or that the program staff do not have the time (14 percent) or the 
expertise (4 percent) “to communicate a new program purpose driven by state 
standards.” 

By coordinating technical assistance, monitoring, and other administrative tasks— and 
thereby reducing their administrative burdens— some managers believe they have more 
time to focus on the complicated issue of improving student achievement. 

Nevertheless, a lack of adequate staff was commonly cited as the reason why programs 
could not focus more on student achievement. Indeed, administrators expressed intense 
frustration about their limited capacity to fulfill administrative tasks, let alone to go 
beyond minimal federal requirements. 

Regarding the implementation of state standards and assessments, several program 
managers explained that they had not paid particular attention to student achievement in 
the past simply because it did not make sense in the absence of state standards and 
assessment systems: what, they asked, would they be measuring in order to assess 
student achievement against? 

Although state administrators said more in 1998 about their efforts to focus program 
operations and activities on raising student achievement, they still expressed reluctance 
to link program success to student achievement. With the exception of most Title I 
administrators, state administrators argued that federal program reporting requirements 
create a disincentive for linking program success to student achievement by continuing 
to focus on process and inputs, rather than on outcomes such as improving student 
performance. 



Uses of Data in SEA Program Management 

As was true in 1996-97, it appeared that the goal of instilling a culture of accountability and 
continual improvement among those administering federally funded programs was unmet. While 
administrators were seemingly collecting or already have at their disposal a great deal of student 
performance and program implementation data, they were not using the data— or, at least, their 
purposes in using the data had little to do with judging the success of federally funded programs in 
raising student achievement. 




Student performance data. 

• Forty-two percent of all state administrators do not require subgrantees to report to the 
state regarding the performance of students participating in their programs. This 
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phenomenon was most common among programs that do not provide direct services to 
students (Eisenhower Professional Development, Safe and Drug-Free Schools and 
Communities, Title VI, and the Technology Literacy Challenge Fund). However, 
substantial numbers of administrators of Migrant Education, Even Start, and Neglected 
or Delinquent programs also said they do not require their subgrantees to submit 
student performance data. 

• The types of student performance data that administrators most frequently said they 
required from subgrantees suggest that increased attention to student achievement 
among federal program administrators may be beginning to take hold: of the 250 
administrators who require districts to submit student performance data, 53 percent said 
they collect data on student performance results from the state assessment; and 42 
percent said they collect performance results from tests other than the state’s 
assessment. 

• While administrators may be collecting student performance data, follow-up survey 
data also show that the number of state administrators who report using the data for any 
purpose has declined since 1996-97: 23 percent acknowledged that they did not use the 
data for any purpose, compared to 15 percent in 1996-97. This was most common 
among administrators of Safe and Drug-Free Schools and Communities and Migrant 
Education programs. 

■ .1 

Program implementation data. 

• Far more state administrators (95 percent) require subgrantees to report program 
implementation than student performance data. The types of program implementation 
data that administrators most frequently said they required from subgrantees were: (1) 
summaries of services rendered; (2) indicators of collaboration with other programs; 
and (3) evidence of varied demographic and ethnic group participation . Most 
administrators said they use program implementation data for (1) reporting to the 
federal government and (2) identifying program services that need to be extended or 
reduced. 

• With the exception of the Title I and Goals 2000 programs, just under half of the state 
administrators (44 percent) reported that program implementation and student 
performance data helped focus program staff on student results and achievement. By 
comparison, 84 percent of Title I administrators and 63 percent of Goals 2000 
administrators reported that the availability of program and student performance data 
helped focus program staff on student results and achievement. 



State Monitoring 



Despite progress since the baseline study, states still had far to go in building new monitoring 
procedures that would communicate a clear message about a new standards-based accountability 
framework. 
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With downsizing of state administrative staffs, monitoring visits continued to be 
infrequent: 1998 survey data showed that more and more programs are visiting fewer 
and fewer subgrantees in a 12-month period. 

As was true in 1996-97, rather than focusing their relatively infrequent monitoring 
activities on districts that might be out of compliance or where student performance 
was low, state program managers most frequently reported using a routine cycle for 
monitoring visits (61 percent). However, 1998 survey data also showed that managers’ 
targeting methods may be starting to change: more administrators said they are using 
information about compliance problems (45 percent versus 36 percent in the earlier 
survey) or student performance (21 percent versus 13 percent) to decide which 
subgrantees to visit, with no particular variation by program. 

Integrated monitoring visits continue to be the trend among federally funded programs; 
about 38 states (up from 27 in 1996-97) conducted some form of integrated monitoring 
visits. However, programs participating in integrated monitoring visits were generally 
visiting fewer subgrantees than were those conducting program-specific monitoring 
visits. 



Federal Monitoring 

Early into the reauthorization period of ESEA, the U.S. Department of Education (ED) began 
to retool its program monitoring system in an effort to better support state and local education reform 
activities. ED established 10 Regional Service Teams (RSTs) charged with the responsibility of 
monitoring and providing technical assistance to state and local education agencies for programs 
authorized and administered out of the Office of Elementary and Secondary Education. As of late 1998 

(when survey data were collected), all but four states had been visited by a federal Integrated Review 
Team (IRT). 

• Among those administrators whose states had been visited by an IRT, many believed 

the visits were somewhat ’ useful (37 percent); fewer respondents considered the visits 
to be “very useful” (18 percent), or “useful” (18 percent), and 19 percent said they 
believed that the visits were “not useful.” Twice as many administrators of Eisenhower 
Professional Development, Title I, Goals 2000, TLCF, and Title VI programs viewed 
the IRT visits as “very useful” than did administrators of other programs. 

Technical Assistance: Are States Working Strategically to Build Local 
Capacity in Support of Standards-Based reform? 



States appeared to make progress in the area of technical assistance. In 1998, program 
administrators showed greater coordination in the content of the technical assistance they provided than 
in 1996-97, and much of that content focused on a standards-based reform agenda. However, as was 
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true in 1996-97, agency downsizing in many SEAs continued adversely to affect the technical assistance 
capacity in federal programs, and interviews with local program administrators indicated that states 
were not meeting their subgrantees’ technical assistance needs. 

• With limited resources, state program administrators continued to make difficult 
choices about where to provide assistance. When asked about technical assistance 
needs that have gone— to a greater or lesser extent— unmet, state administrators most 
often said they are unable to get to every district that needs or requests help or that they 
are unable to provide sustained assistance or follow-up. 

• Increasing numbers of state administrators (81 percent, up from 72 percent in 1996-97) 
were relying on districts to know when they need help and how to ask for it, rather 
than actively assessing local need for assistance in implementing their programs. 

• In site visits to 16 districts in eight states, our interviews with local program 
administrators generally confirmed state managers’ worst fears: states are not meeting 
their subgrantees’ technical needs. Most local administrators refer to the tremendous 
turnover their SEAs have experienced in the past several years and mourn the loss of 
institutional knowledge; some feel they are having to spend time training their state 
administrators to do their jobs. 



Performance Indicators: Are States Collecting and Using Indicators Data to 
Inform Program Performance? 

Findings from the baseline study showed that program performance indicators were not a 
particularly common means by which program administrators assessed and improved program success. 
Since that study was conducted, the U.S. Department of Education has developed— as mandated under 
the Government Performance and Results Act— a set of program performance indicators intended to 
inform Congress, the Department, and the nation about the effectiveness of federal elementary and 
secondary programs. 



State-Developed Program Performance Indicators 

Program performance indicators appear to be on the rise among federally funded programs. In 
1998, 55 percent of state administrators reported that their program either had developed (26 percent) 
or was in the process of developing (29 percent) performance indicators; in 1996-97, the overall 
percentage was just 41 percent. 

• Performance indicators were found in significant majorities in state offices 

administering Safe and Drug-Free Schools and Communities (85 percent) and 
Eisenhower Professional Development (84 percent) programs. 
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Among the 26 percent of state administrators who reported having already developed— 
as opposed to being in the process of developing— performance indicators, most 
reported that they were using the indicators to inform their work and to facilitate cross- 
program coordination and planning. 



Federal Program Performance Indicators 

In 1998, state administrators were asked whether they: (1) were aware of the program 
performance indicators developed by the U.S. Department of Education and (2) collected and used the 
federal performance indicator data. 

• About 62 percent of all state administrators reported being aware of the fact that the 
U.S. Department of Education had developed a set of performance indicators for their 
respective programs. Administrators of Eisenhower Professional Development, Safe 
and Drug-Free Schools and Communities, and the Technology Literacy Challenge 
Fund were well above average in terms of knowing about the existence of the federal 
program performance indicators. 

• Of the 267 program administrators who were aware of the federal program 
performance indicators, 217 or 81 percent also collected some or all data on them. 
Administrators of Safe and Drug Free Schools and Communities and the Technology 
Literacy Challenge Fund reported both being aware of and collecting these data more 
than any other program (100 percent and 92 percent of administrators collected data, 
respectively). 



Of the 217 program administrators who collect some or all data related to the federal 
program performance indicators, most administrators reported using the data to report 
to federal officials (81 percent). 



Conclusions 



This study found that in 1998— four years after reauthorization of ESEA and four-and-a-half 
years after the authorization of Goals 2000-states had made significant progress in implementing the 
legislation in a number of areas. 



The vast majority of state administrators in 1998 perceived flexibility in the legislation, 
and many reported using that flexibility to help districts find ways to use federal 
program resources to meet local needs. 



In 1998, administrators were more likely to recognize linkages between program 
purposes and student achievement and to acknowledge that supporting standards-based 
reform is part of their programs’ purpose. These changes seemed due largely to efforts 
to coordinate program administration and operations across federal programs and the 
long-awaited implementation of state standards and assessment systems. 
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• Despite progress since the baseline study, states still had far to go in building new 
monitoring procedures that would communicate a clear message about a new, 
standards-based accountability framework. In addition, although administrators seemed 
to have access to a great deal of data about student performance and program 
implementation, they were most often using the data for purposes other than judging 
the success of their programs in raising student achievement. 

A great deal of organizational learning was called for in order to respond to the challenges 
presented by the new and reauthorized laws placed before the SEAs in 1994. The baseline and follow- 
up studies point to progress not only in initiating new administrative routines but also in developing a 
new outlook on program purposes and priorities. Although state administrators were not uniformly 
adhering to an agenda of standards-based, data-driven reform in 1998, these studies provide evidence 
that they had moved in that direction in some respects and might be more strongly encouraged and 
helped to do so in other respects. 
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I. Introduction 



The Goals 2000: Educate America Act and the Improving America’s Schools Act (IASA), 
which amended the Elementary and Secondary Education Act (ESEA), were enacted in 1994. 

Together they were designed to provide a comprehensive system of support for state and local 
education reform initiatives that would enhance children’s educational achievement. This federal 
legislation promoted the use of federal funds to support systemic, standards-based approaches to 
improving the quality of teaching and learning. Specifically, these federally supported elementary and 
secondary education programs set an ambitious agenda of policy changes, including supporting states in 
the development of: 

• Challenging state standards of curriculum content and student performance. 

• High-quality student assessment (and accountability) systems that are aligned with 
challenging state content and student performance standards. 

• Sustained, intensive professional development aligned with challenging state standards. 

Goals 2000 and the programs reauthorized under IASA also promoted a more coordinated, 
coherent approach to program administration; the barriers between categorical programs were reduced. 
The laws allowed state administrators of federal programs to coordinate and consolidate their 
administrative functions so as to minimize the burden and cost and thereby redirect their programs to 
support broader state policy initiatives, such as the implementation of standards. These laws also 
offered greater decision-making authority and flexibility to local administrators and teachers in 
exchange for greater responsibility for student performance. 

State education agencies (SEAs) play a crucial role in implementing the new laws: they are a 
primary source of information and guidance for local school districts; and the states have the lead role 
in setting academic standards and deciding how to assess student progress. Goals 2000 and the 
reauthorized ESEA encouraged state administrators of federal programs to use new approaches in 
program management— to communicate a more concerted focus on improving students’ chances of 
meeting high standards, and to pull administrative operations together across categorical programs. 

This study, conducted under contract with the Planning and Evaluation Service in the Office of the 
Under Secretary, U.S. Department of Education (ED), focuses on the work of these key administrators 
at the state level. It follows up on baseline information collected during late fall 1996 and early winter 
1997, analyzing the ways in which state administrators of federally funded programs have continued to 
respond to the new legislative framework. Data for the follow-up study were collected in summer and 
fall 1998. 



Findings from the Baseline Study 



In the first two years of implementation of Goals 2000, the reauthorized ESEA programs, and 
the Education for Homeless Children and Youth Program, SEA administrators of these federal 
programs had taken major steps toward cross-program communication within their own agencies, and 
many program administrators had also communicated a message of broad program change to their local 
school districts. Effects were beginning to emerge, although they were not as large as some might 
wish. Where early implementation fell short of the original federal vision was in the depth and 
intensity of program administrators’ work around standards-based educational improvement. (Findings 
presented in this section were collected in the 1996-97 survey; results of that survey are also 
summarized in Anderson and Turnbull [1998].) 

• All SEAs made noticeable changes to their procedures in implementing the 



reauthorized programs. Consolidated plans were almost universal, and the planning 
process had helped inform administrators about each others’ programs; in many cases, 
planning had also given them new ideas about ways to work together. This 
collaborative work was beginning to result in the acceptance, albeit often on a pilot 
basis, of consolidated plans from local school districts and integrated monitoring visits 
across programs. 

Most program administrators gave little evidence that they were attending to student 
performance. Having largely dismantled their old monitoring systems, few were 
sending a strong message to their districts that accountability for student performance 
would replace the compliance monitoring of the past. 

Compliance monitoring was being replaced with technical assistance— primarily offered 
to districts that knew when and how to ask for it. 



In short, new procedures for program administration (such as consolidated state planning) were 
a force behind a good deal of change. The program administrators also reported that downsizing and 
other SEA reorganizations had prompted changes in program management. Largely missing from the 
understanding of most program administrators, however, was an urgency to organize their day-to-day 
work around aligning program services and operations with the expectations for students’ academic 
performance embodied in state content and performance standards. 
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Design Changes in the Follow-up Study 



While the study purposes and research questions remain largely the same, the follow-up study 
differs in some important respects from the baseline study of state implementation of federal elementary 
and secondary programs. First, because a national evaluation of the Education for Homeless Children 
and Youth Program is currently underway and includes telephone interviews with all the state 
coordinators of the Education for Homeless Children and Youth Program, that program was not 
included in the follow-up state implementation study. Second, the follow-up state implementation study 
included site visits to eight rather than 13 states, and included interviews with program administrators 
at both the state and district levels (two districts per state). Third, the follow-up study had a special 
focus on the implementation of the Education Flexibility Partnership Demonstration Program (Ed-Flex) 
among the 12 states that received Ed-Flex authority under the Goals 2000: Educate America Act. The 
follow-up study also investigated state-level operations and effects of Title III, the Technology Literacy 
Challenge Fund. Finally, the follow-up study gathered information on the extent to which states are 
collecting data that would inform the program performance indicators developed by ED under the 
Government Performance and Results Act. The data were collected in summer and fall 1998. 

I 

The Nine Federal Programs Studied 



This study focuses on Goals 2000 and eight programs under the 1994 Improving America’s 
Schools Act, which amended the Elementary and Secondary Education Act. The programs vary in 
purpose, size, and funding arrangements, but each one has given the states a key role to play in 
communicating program purposes and procedures to local districts. The programs included in this 
study are (in order of size): 1 



Title I of the Elementary and Secondary Education Act (ESEA) of 1965: Helping 
Disadvantaged Children Meet High Standards; Part A: Improving Basic Programs 
Operated by Local Educational Agencies (amended in 1994). Supports local 
educational agencies in improving teaching and learning to help low-achieving students 
in high-poverty schools meet the same challenging state content and performance 
standards that apply to all students. Promotes effective instructional strategies that 
increase the amount and quality of learning time for at-risk children and that deliver an 
enriched and accelerated curriculum. Also expands eligibility of schools for 
schoolwide programs that serve all children in high-poverty schools; encourages 



1 The IASA program descriptions are taken from U.S. Department of Education, Cross-Cutting Guidance for the 
Elementary and Secondary Education Act (Washington: September 1996); the descriptions of Goals 2000 and 
Education for Homeless Children and Youth are taken from the statutes. 
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school-based planning; establishes accountability based on results; promotes effective 
parental participation; and supports coordination with health and social services. 

Type of Assistance: Formula grants 

FY 1998 Appropriation: $7.5 billion 

Title IV, ESEA: Safe and Drug-Free Schools and Communities. Supports Goal Seven 
of the National Education Goals by encouraging comprehensive approaches to make 
schools and neighborhoods safe and drug-free. Provides funds to governors, state 
education agencies (SEAs), local education agencies (LEAs), institutions of higher 
education, and nonprofit entities for a variety of drug and violence prevention 
programs. 

Type of Assistance: Formula grants to state education agencies and local 

education agencies; formula grants to governors who 
make discretionary awards 
FY 1998 Appropriation: $531 million 

Title III, Goals 2000: Educate America Act: State and Local Education Systemic 
Improvement. Seeks to “improve the quality of education for all students by improving 
student learning through a long-term, broad-based effort to promote coherent and 
coordinated improvements in the system of education throughout the nation at the state 
and local levels” (Sec. 302). 

Type of Assistance: Formula grants to states; discretionary grants to 

districts. 

FY 1998 Appropriation: $466 million 

Title III, Subpart 2, ESEA: Technology Literacy Challenge Fund. Provides resources 
to speed the implementation of statewide strategies designed to enable all schools to 
integrate technology fully into school curricula, so that all students become 
technologically literate with the reading, math, science, and other core academic skills 
essential for their success in the 21st Century. 

Type of Assistance: Formula grants to state education agencies; 

discretionary grants to districts 
FY 1998 Appropriation: $425 million 

Title VI, ESEA: Innovative Education Program Strategies. Provides broad support 
for activities that encourage school reform and educational innovation. 

Type of Assistance: Formula grants 

FY 1998 Appropriation: $350 million 

Title II, ESEA: Dwight D. Eisenhower Professional Development Program. 
Concentrates on upgrading the expertise of teachers and other school staff to enable 
them to teach all children to challenging state content standards. Supports sustained 
and intensive high-quality professional development, focused on achieving high 
performance standards in mathematics, science, and other core academic subjects. 



Type of Assistance: 

FY 1998 Appropriation: 



Formula grants 
$335 million 



Title I, ESEA, Part C: Education of Migratory Children. Supports educational 
programs for migratory children to help reduce the educational disruptions and other 
problems that result from repeated moves. Helps provide migratory children with the 
same opportunities as other children to meet challenging state content and performance 
standards. Targets efforts on the most mobile children, whose schooling is most likely 
to be disrupted. 

Type of Assistance: Formula grants 

FY 1998 Appropriation: $305 million 

Title I, ESEA, Part B: Even Start Family Literacy. Improves the educational 
opportunities of low-income families by integrating early childhood education, adult 
literacy or adult basic education, and parenting education into a unified family literacy 
program. 

Type of Assistance: Formula grants to state education agencies, which in 

turn make discretionary grants to partnerships of local 
education agencies and nonprofit community-based 
organizations or other nonprofit organizations; federal 
discretionary grants for projects that serve migratory 
children and their families, Indian tribes, tribal 
organizations, the outlying areas, and a project in a 
prison housing women and preschool-aged children; 
and to states for statewide family literacy initiatives. 

FY 1998 Appropriation: $124 million 

Title I, ESEA, Part D: Prevention and Intervention Programs for Children and 
Youth Who Are Neglected, Delinquent, or At-Risk of Dropping Out, Subpart 1. 
Extends educational services and learning time in state institutions and community-day 
programs for neglected and delinquent children and youth. Encourages smooth 
transitions to enable participants to continue schooling or enter the job market upon 
leaving the institution. 

Type of Assistance: Formula grants to state education agencies; 

discretionary grants to state agencies and local 
education agencies 

FY 1998 Appropriation: $39 million 



Study Purposes and Research Questions 



The evaluation’s purposes were derived from ED’s larger framework of data collection and 
analysis for the National Assessment of Title I (mandated in Sec. 1501 of the Improving America’s 
Schools Act) and a comprehensive evaluation of federal support for elementary and secondary reform 
(Sec. 14701). In consultation with an Independent Review Panel of state and local educators, 
researchers, and other citizens, ED decided to focus on the following aspects of program 
implementation at the state and local levels: high academic standards for all children; assessment and 
evaluation; support for enriching curriculum and instruction; flexibility coupled with accountability for 
student performance; and targeting of resources. 

Cutting across these topical areas of focus are this study’s questions about administrative 
processes at the state level: 

• How are state program managers currently implementing the law’s provisions? 

• How has implementation changed when compared with state practices under the 
predecessor programs? 

• What federal and state factors underlie these changes? 

This study, then, focused on changes in program administration after the 1994 reauthorization 
of federal programs under ESEA and the enactment of Goals 2000. It explored state-level progress in 
the years following the 1994 reauthorization toward administering the federal programs in ways that: 

• Make use of increased flexibility across programs. 

• Make programs more accountable for student performance. 

• Support improvements in teaching and learning. 

These elements of state program administration form the organizing structure for this report: each of 
the next three chapters presents findings related to one of the above elements; a fourth chapter 
discusses the development and use of state and federal program performance indicators as a specific 
approach to program assessment and improvement; and a concluding chapter discusses overall trends 
in state administration of federal programs. 




6 



26 



Study Methods 



Surveys were administered during late summer and fall 1998 (approximately four years after 
the reauthorization of ESEA, four-and-a-half years after the authorization of Goals 2000, and two years 
after the baseline study data were collected) in all 51 state education agencies (including the District of 
Columbia) to state-level managers of each of nine federal programs, plus administrators knowledgeable 
about the Education Flexibility Partnership Demonstration Program or “Ed-Flex” (henceforth referred 
to as “Ed-Flex administrators”), for a total of 468 possible respondents. As was true of the baseline 
study, respondents did not complete paper-and-pencil surveys. Instead, each follow-up survey was 
administered by telephone as a personal interview with standard questions, some of them closed-ended 
and some open-ended. The interviewers recorded all responses on written forms. Responses to closed- 
ended questions were tabulated; responses to open-ended questions were coded for tabulation as well as 
yielding more elaborated information. 

To explore cross-cutting matters such as state procedures in planning or technical assistance 
under all nine programs, we again created a core survey to be administered to state-level managers of 
all the programs. By asking so many identical questions across programs, the survey enabled us to 
present comparative findings throughout this report. In some cases, however, cross-program 
differences reflect real policy differences in program purposes and approaches; this study’s approach 
should not be construed as implying that all programs ought to be administered identically. For all nine 
programs, tailored questions about performance indicators, mandated targeting, assessment, and other 
administrative procedures were added to the core survey. Finally, a separate survey was developed 
and administered to the relevant program coordinators in the 12 states selected to participate in the “Ed 
Flex” program. 

Out of a possible 468 surveys, 447 were completed, a response rate of 96 percent. 



Survey Respondents 

To identify our respondents for the follow-up study, we again called each individual identified 
by ED as the state contact person or coordinator for each of the nine programs in each of the 50 states 
plus the District of Columbia. We asked that person if he or she would be able to answer questions 
about cross-program coordination, subgrant applications, project monitoring, accountability, and 
technical assistance under that program. We also asked for the name of anyone else in federal program 
administration at the state level whom we should interview regarding these issues. 
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As was true in the baseline study, the respondents we ultimately identified and interviewed 
were managers who: (1) often administered more than one federal or state program; (2) varied in the 
length of their administrative experience; (3) worked out of a variety of offices and divisions that might 
or might not be housed in the state education agency (SEA); and (4) might supervise or be supervised 
by other respondents to this survey. In addition, the Title I coordinator was often the respondent for 
both the Title I and Neglected or Delinquent programs; respondents for the Eisenhower Professional 
Development and Title VI programs were sometimes the same person. 
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II. Flexibility: Do States and Districts Continue to Experience 
New Latitude in Implementing the Law? 



According to the reauthorized Title I, “Decentralized decision-making is a key ingredient of 
systemic reform. Schools need the resources, flexibility, and authority to design and implement 
effective strategies for bringing their children to high levels of performance” [Sec. 100 1(c)(8)], To help 
states in their efforts to raise the academic achievement of all students to high standards— recognizing 
that there may be many ways to do so— the Goals 2000 Act and the IASA amendments attempted to 
offer states greater flexibility in the use of federal program resources and in the administration of 
federal program services. This flexibility for states mirrored the flexibility that policymakers sought to 
offer to schools and school districts. Believing that schools, districts, and states should have the 
freedom to do what it would take to raise students’ achievement, unencumbered by administrative 
barriers, the laws encouraged cross-program planning as well as coordination of administrative and 
operational activities, consolidation of administrative funding, and consolidated plans or applications 
from local school districts or other subgrantees. 

In this chapter, we discuss the ways in which state program administrators have continued to 
make use of the flexibility provisions in the new legislative framework and the extent to which they 
have moved beyond attention to procedural provisions— such as consolidated planning and cross- 
program communication— and begun focusing on the task of using the flexibility provisions in the 
legislation to align program services and operations to support state content and performance standards 
and thereby improve student achievement. Specifically, this chapter discusses the extent to which 
program administrators (1) perceived that their flexibility had increased in the four years after the 
reauthorization of ESEA, (2) were using each of several administrative provisions in the legislation and 
how using those provisions affected their work, and (3) encouraged program coordination at the local 
level. 

States’ Administrative Flexibility 

“Flexibility” throughout the intergovernmental system is a watchword of IASA. Four years 
after authorization, did more state administrators of IASA programs think their own flexibility had 
increased than did in 1996-97 when the baseline study was conducted? And what more had they done 
since 1996-97 to avail themselves of various options intended to increase flexibility and cross-program 
coordination? 
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Perceptions of Increased Flexibility 



Two-thirds of state administrators of ESEA programs believed that their own flexibility had 
increased in the four years after reauthorization. In fact, the majority of administrators of all seven 
ESEA programs who were asked to compare current administrative flexibility to that which they had 
prior to the 1994 reauthorization 2 reported experiencing increased flexibility. Compared to survey data 
collected in 1996-97, this finding shows a significant change in administrators’ perceptions of 
flexibility, especially in the programs other than Title I (the only one in which a majority of 
administrators had reported an increase in administrative flexibility at the time of the earlier survey). 
Only seven percent of respondents in 1998 reported that they found no increase at all in administrative 
flexibility since the reauthorization; almost one-third of respondents had made that claim in 1996-97 
(Table 1). A number of administrators still said in 1998 that their program has always offered 
flexibility: “There is no need for additional flexibility because it is already there” (Eisenhower 
Professional Development). Nevertheless, there was an overall rise in perceptions of increased 
flexibility. This may result from the fact that the legislation was no longer new; administrators had had 
time since 1996-97 to familiarize themselves with the reauthorized ESEA and better understand and act 
upon the flexibility it affords them. This notion is borne out by the fact that greater numbers of 
administrators in 1998 reported using more of the new administrative procedures that the reauthorized 
ESEA affords them (i.e., for program operations, local applications, monitoring, professional 
development, or technical assistance) than did in 1996-97. Specifically, 54 percent of program 
managers for Title I, Even Start, Migrant Education, Neglected or Delinquent, Eisenhower 
Professional Development, Safe and Drug-Free Schools and Communities, and Title VI reported using 
four or five new administrative procedures in 1998, compared to 37 percent of program managers 
reporting for those same programs in 1996-97. 

Changes in recent years in the kinds of signals that the U.S. Department of Education is 
sending to state administrators also appear to have contributed to perceptions of increased flexibility, as 
the following comment suggests: 

From the federal level, there seems to be more flexibility than at the state level, for whatever 
reason. Federal guidelines now, more often than not, emphasize flexibility: “I’m not going to 
tell you how to write your plan, but here is a set of questions and answers.” (Goals 2000) 



2 Administrators of Goals 2000 and Technology Literacy Challenge Fund (TLCF) programs did not answer the 
question about flexibility prior to reauthorization because these programs did not exist prior to the 1994 
reauthorization. 
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Table 1 



Extent to Which State Administrators Find the Reauthorized Legislation 
Gives Them More Administrative Flexibility Than They Had 
Before the Reauthorization, by Program 
(N=332) 1, 2 



Taking into account all of your offices’ responsibilities under this program, to what extent do you find that this 
legislation gives you more administrative flexibility than you had before the reauthorization? 



Number of State Administrators Reporting 
That Reauthorization Gives Them 
Administrative Flexibility: 



Federally Funded 
Education Programs 


To a 

Considerable 

Extent 


Somewhat 


Not at All 


No Change 


Don't 

Know 


Title I, Part A (N=49) 


24 


19 


0 


3 


3 


Safe & Drug Free Schools 


and Communities (N=47) 


10 


19 


3 


11 


4 


Title VI (N=48) 


14 


16 


7 


7 


4 


Eisenhower Professional 


Development (N = 49) 


15 


18 


4 


7 


5 


Education of Migratory 


Children (N=45) 


9 


19 


3 


13 


1 


Even Start Family 


Literacy (N=49) 


9 


18 


2 


16 


4 


Programs for Neglected or 
Delinquent Children and 
Youth (N=45) 


14 


13 


3 


12 


3 


TOTAL 


95 


122 


22 


69 


24 



1 Respondents were administrators of the seven reauthorized ESEA programs (excluding Goals 2000 and TLCF, 
which are new, not reauthorized legislation). 

2 Programs are listed in order of appropriations size throughout this document. 
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In addition, it appears that administrators are more aware of the waiver option than in 1996-97 and 
recognize it as another vehicle through which they can obtain needed flexibility; this was especially true 
among program managers in states participating in the Education Flexibility Partnership Demonstration 
Program (Ed-Flex). 3 The most frequent response by state administrators to the question about 
additional flexibility needed was: “We are an Ed-Flex state; we don’t need more flexibility” or “We 
are an Ed-Flex state; if we need more flexibility, we’ll take it.” 

When asked whether they need additional flexibility, most administrators reported that the 
legislation affords them ample flexibility; nevertheless, some cautioned that perhaps they have been 
given too much flexibility, thereby exposing programs to a variety of problems. Administrators 
commented: 



The flexibility makes the program easy to administer and well-liked. It can be adapted to help 
meet state and local goals. However, the flexibility has probably made it more vulnerable to 
politics at the state and federal levels. (Title VI) 

I believe it is flexible enough. I think maybe there could be a little less flexibility to make sure 
[that] targeted people are being served. (Migrant Education) 

Flexibility is a double-edged sword. We can have so much flexibility that it becomes 
meaningless — you have no guidance then. (Eisenhower Professional Development) 

Finally, some program administrators explained that although they may have the flexibility they 
need from the federal level to successfully administer their programs, state-level policies and practices 
can curtail the flexibility that the reauthorized ESEA affords them: 



The federal law is sufficiently flexible, but the state is inflexible in some areas. The governor 
has a lot of influence over the program. When Even Start funds subgrantees, its 
recommendations are usually overridden by the governor’s office. As a result, the federal 
process that safeguards neediest programs often gets sacrificed. (Even Start) 

If our state legislature allowed us to manage funds in a different way (i.e., consolidate) it would 
be great. Our state is our own barrier. (Title VI) 

In the state, there is too much control by the state legislature. We end up not able to make 
decisions based on educational concerns because the legislature makes decisions based on 
political concerns. (Goals 2000) 



3 Established by the Goals 2000: Educate America Act, Ed-Flex gives participating states the “power to waive 
requirements of certain federal education programs, including Title I and the Eisenhower Professional 
Development Programs” in exchange for increased accountability for results. Without Ed-Flex, states and school 
districts may ask the secretary of education to waive these requirements; with Ed-Flex, states have the authority to 
make those decisions at the state level. [Ed-Flex Fact Sheet. ED, 1996b] 
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The perception of increased flexibility did vary by state, but again, not by the sizable numbers 
reported in 1996-97. That is, in 1998, there were 34 states in which most of the ESEA program 
administrators (four or more of seven) reported at least some increase in their flexibility; in 1996-97, 
this was true for only 12 states. Moreover, in 1998, there were only three states in which just one 
administrator or none reported an increase in flexibility; in 1996-97, this was true in 11 states. In the 
12 Ed-Flex states, 74 percent of all administrators reported an increase in flexibility (in 1996-97, it was 
47 percent); in the non-Ed-Flex states, the figure was 61 percent (in 1996-97, it was 38 percent). 



What Did “State Flexibility” Mean to State Administrators? 

In describing the administrative flexibility available to them since reauthorization, state 
program managers expressed two major themes: (1) the opportunity to coordinate and collaborate with 
other federally funded programs and (2) the opportunity to support local reform plans. The first theme 
was also prominent in responses collected during the baseline study; the second, however, grew in 
frequency, suggesting that states may have moved further along in their thinking about the ultimate 
purpose of their programs: to support local reform efforts. In addition, program managers also 
pointed to several areas in which they felt they needed additional flexibility, although most of their 
concerns were related to program-specific legislative requirements. A sizable number of state 
administrators talked about the need to consolidate program reporting requirements to more closely 
track with the goals and objectives set forth in their consolidated state plans. 

Coordination and collaboration among administrators. As was true in 1996-97, in responses 
to a variety of open-ended survey questions, many respondents talked about the opportunity to 
coordinate and collaborate with other program administrators. The following comments were typical: 

Flexibility has helped us coordinate and integrate with the other programs; it has reduced 

barriers and territorial issues to some degree. (Safe and Drug-Free Schools and Communities) 

Flexibility has increased program planning and coordination among other programs. 

(Eisenhower Professional Development) 

In some cases, as was true in 1996-97, respondents took this theme a step farther and cited 
benefits to students that could result from their coordination and collaboration: 

Having the flexibility to join with other programs and to use funding in a blended fashion 

provides integrated service delivery which improves performance. (Even Start) 

The flexibility has allowed us to work across programs. The SEA no longer distinguishes 

between programs— it focuses more on student needs. (Title VI) 
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Finally, many state administrators cited the decrease in the level of duplicated effort among 
federal programs as one of the more practical benefits associated with cross-program coordination and 
collaboration: 



It is easier to look at the consolidated application and consolidated reviews. It conserves effort 
and reduces duplication. (Eisenhower Professional Development) 

It is easier to do some things because of the additional flexibility. For example, in the past we 
had to send separate approval letters for each program in a district; now we can just send one 
letter. (Title I) 

Supporting local reform. When state program managers were asked to elaborate on the effects 
of new flexibility at the federal and state levels on their programs, many program managers described 
efforts to help districts find ways— sometimes creative ones— to use federal program resources to meet 
local needs rather than using administrative flexibility to simply administer programs within an explicit 
framework of program regulations and requirements. These responses signaled an unmistakable shift 
in outlook and orientation from those made in 1996-97 when, in response to a similar question, 
respondents were more apt to cite improvements in state-level functions, such as flexibility to 
coordinate and collaborate with other program administrators and the opportunity to consolidate state 
administrative funding. As the following comments suggest, however, few administrators mentioned 
the relationship between flexibility and improvements in program services or student outcomes: 



Goals 2000 is rather non-prescriptive, and the flexibility in terms of the performance of the 
program is that local school districts have truly been able to identify and respond to local needs 
versus responding to either state or federally imposed requirements. That’s fairly 
unprecedented for a federal program to be that non-prescriptive. (Goals 2000) 

The fact that flexibility is extended to LEAs and allows them to design a program that meets 
their needs is great. We’re finding that LEAs are having a tremendous impact because they 
have designed something that fits their needs; they’re not having to force things (i.e., some 
externally developed agenda) on communities. (TLCF) 

The new flexibility in the law enhances productivity and accountability. The law gives us the 
authority to really adapt programs to local needs. It is very accommodating. (Migrant 
Education) 

Flexibility has helped shift funds to areas with greater needs and enabled programs to better 
meet locally identified and locally defined needs. (Title VI) 

Consolidating federal reporting requirements. Several program managers expressed 
frustration at not being afforded the same flexibility in responding to federal reporting requirements 
that they enjoy with respect to program planning and funds consolidation; some argued that ED’s 
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program-specific reporting requirements work to dissuade program managers from coordinating 
program services and activities as well as causing states to impose undue reporting burdens on districts. 
The following comments were typical: 



Require the same data from all federal programs— we need consolidated reporting to go with 
consolidated planning. We need a set of consistent regulations; there should be consistent data 
that will be accepted in all the programs. The regulations have to be reviewed to determine 
what’s consistent in them. We go crazy with all the individual inquiries. (Title I) 

It’s very difficult to evaluate federal programs. With reauthorization, we were asked to 
consolidate programs, but ED didn’t respond by consolidating the evaluation process, so we’re 
still going out to districts asking for program information. The federal government wanted it to 
be less burdensome, but with evaluation, we’ve become more burdensome. We’ve worked— 
and the Comprehensive Center has worked— on developing a consolidated evaluation 
instrument. But as it stands now, we make a lot of unfair demands on districts. (Eisenhower 
Professional Development) 

The SEA is still in the evolving process of trying to coordinate all functions, all decision- 
making, and we’re not quite there yet. We’ve made a lot of progress, but we do at times 
impose separate requirements on districts, and that is really driven by ED imposing separate 
requirements on us, and we, in turn, do it to the districts. We’re all trying to align, but we’re 
not quite there yet. (Goals 2000) 

As some state administrators suggested, by not consolidating federal reporting requirements, ED may 
be undermining its efforts to encourage state and local coordination and collaboration in support of 
standards-based reforms: 

What would be helpful is a single integrated program performance report that cuts across all 
these sources of funds and focuses in on the impact that these resources have had on 
implementing a system of standards and assessments. It would make sure we had developed it 
together at the state level, as opposed to having separate reports. (Goals 2000) 

The biggest thing we need, if ED is really pushing for integration, consolidation, and 
leveraging funds, is to look at differences at the federal level among reporting requirements for 
each program. Each program has its own evaluation at the federal level— Eisenhower 
Professional Development, Title IV, Title VI. Those evaluations force reporting out by 
program and that tends to be self-defeating. If ED could develop a consolidated reporting 
mechanism for programs, it would facilitate what it wants to happen in terms of integration. 
(Goals 2000) 



State-Level Program Coordination 



According to ED’s cross-cutting guidance regarding state implementation of the amendments to 
ESEA, consolidated planning would enable states “to plan how to use all of their federal funds to 



support overall state goals” (ED, 1996a; p.7). In 1996-97, we learned that almost all state 
administrators of the programs included in this present study (with the exception of the Technology 
Literacy Challenge Fund (TLCF), which was not operating in 1996-97) participated in the development 
of their state’s consolidated plan, but that only a few states had achieved an across-the-board 
consolidation of administrative funds. What we did not know was to what extent consolidated planning 
was translating into coordinated program administration and operations; many administrators’ responses 
suggested that they were just planning for planning’s sake, because they did not speak about the more 
substantive outcomes of planning. Moreover, we did not know whether and to what extent 
administrators were organizing program administration and operations around supporting state goals or 
whether consolidating administrative funds was translating into increased administrative and operational 
capacities among programs. Accordingly, we followed up in all these areas in 1998. This section 
discusses the types of flexibility provisions states used in 1998 — including consolidated planning, 
consolidation of administrative funds, and Ed-Flex waivers— and state administrators’ perceptions of the 
effects of their use. 



State Coordination of Program Services and Operations 

Most state administrators (81 percent) reported conducting specific administrative or 
operational activities in coordination with other federally funded education programs (Table 2). The 
types of coordinated activities most frequently cited were: (1) providing technical assistance to districts 
and schools (89 percent); (2) holding local application and planning workshops (79 percent); (3) 
monitoring local projects (70 percent); and (4) decision-making with respect to allocating program 
resources to districts and schools (60 percent). Far fewer programs collaborated around collecting 
student (40 percent) or program (41 percent) performance data or in conducting local needs 
assessments (44 percent) (Table 3). 

Titles I, Eisenhower Professional Development, Safe and Drug-Free Schools and Communities, 
and Title VI tended to coordinate their administrative and operational activities together, as did Goals 
2000 and TLCF. The other, smaller programs, including Even Start, Migrant Education, and 
Neglected or Delinquent, did not necessarily form their own coordinated unit; rather, they were either 
included or excluded— in no consistent pattern— from the larger group of coordinated programs. Even 
Start tended to be the biggest outlier with respect to program coordination, often citing Adult Education 




16 



36 



Table 2 

State Administrators Who Report Conducting Administrative or 
Operational Activities in Coordination with Other 
Federally Funded Education Programs, by Program 
(N=434) 



Since January of 1997, has your program conducted specific administrative or operational activities (e.g., monitoring local 
prnjpctc, providing technical assistance to districts and suberantees, etc.) in coordination with other federally funded 
education programs? 


Federally Funded Education Procrams 


Number of State Administrators 


Title I, Part A (N=49) 


45 


Safe and Drug- Free Schools and Communities (N=48) 


40 


Goals 2000 (N=46) 


38 


Technology Literacy Challenge Fund (N=49) 


34 


Title VI (N=48) 


42 


Eisenhower Professional Development (N=50) 


46 


Education of Migratory Children (N=47) 


35 


Even Start Family Literacy (N=50) 


37 


Programs for Neglected or Delinquent Children and Youth (N=47) 


36 


TOTAL 


353 
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Types of Administrative or Operational Activities State Administrators 
Report Conducting in Coordination with Other Federally Funded 
Education Programs, by Program 
(N=353) 





and Early Childhood Education programs as its predominant collaborating partners. As one Even Start 
coordinator explained: “We don’t collaborate with other federal programs because it makes more 
sense to collaborate with state and local early childhood education programs.” 

Although states reported doing a great deal more to coordinate administrative and operational 
activities than was evident in 1996-97, it is important to note that it took a long time to get to this point, 
and many programs are still not participating in coordinated activities. Evidently, state administrators 
are still “learning about each other’s programs” and attempting to identify ways that they can work 
together; they have only just begun coordinating certain administrative and operational activities, and 
only with a core group of programs (typically, Titles I, VI, Eisenhower Professional Development, and 
Safe and Drug-Free Schools and Communities): 

We are trying to pilot test how well school districts will adapt to consolidated programs. We 
did not want to throw them all on them at one time. We may include other programs in the 
future, depending on the success of the pilot test. (Title VI) 

The state is still in the process of change and has only started coordinating services and 
activities with a few programs. We will add programs because we are all under the same 
associate superintendent. (Eisenhower Professional Development) 

We are just beginning to coordinate programs. These were the first ones. . . we are still in the 
process of reorganization. (Goals 2000) 

To really coordinate well takes time and we’re just in the first stages of it. There’s just not 
time to coordinate. We have plans to, but we’re not there yet. It takes time to figure out how 
to coordinate. (Eisenhower Professional Development) 

When asked why administrative and operational activities are coordinated with some programs 
and not with others, state managers often explained that the programs that were included in the 
consolidated state plan are usually the ones included in broad-based coordination efforts. Other 
responses clustered into the following three categories: (1) the organization of the SEA impedes efforts 
to include all programs in coordinated activities; (2) some programs are more difficult to work with 
because of the targeted populations they serve or the discretionary nature of their grants award system; 
and (3) not all federal programs share the same focus. 

Organizational structure of the State Education Agency (SEA). Many administrators 
explained that no matter how desirable it might be to coordinate administrative and operational 
activities with other federal programs, the organizational structure of the SEA simply stands in the way: 

Things are spread around a fairly large area. We are close to some programs but not others. 
(Goals 2000) 



Not all programs are in the same division. I coordinate with ones I have authority over. 
Federal programs are under different associate superintendents and different directors. When 
you try to coordinate efforts, you rely on supervisors to relay messages to their people. Some 
folks are pretty [protective] about their programs. They see any kind of coordination as a 
dilution of their program and they fight it tooth and nail. (Title I) 

Working across divisions is difficult because of the different principles that guide each division. 
(Title VI) 

Others explained that the state political context within which they operate federal programs also has 
some bearing on the extent to which they are able to conduct cross-program coordination: 



The culture of the state bureaucracy has not been to value collaboration and coordination 
because everything was an elevator going up and down, not across, because it was driven by 
either federal or state funding. Things were seen as needing to be departmentalized in that 
way, and changing mindsets is very difficult to do. (Goals 2000) 

The role of smaller federal programs in coordinated services and activities. Although 81 
percent of the respondents reported conducting administrative or operational activities in coordination 
with other federally funded education programs, there was markedly less participation in those 
coordinated activities among administrators of the smaller or discretionary grants programs— Migrant 
Education, Even Start, and Neglected or Delinquent. Roughly one-fourth of the administrators for 
these programs said that they were not coordinating any administrative or operational activities with 
other federally funded education programs. In addition, a sizable percentage of administrators for the 
Technology Literacy Challenge Fund (31 percent) were non-participants in cross-program coordination 
of services and activities. Many of the TLCF program administrators explained that they have not yet 
participated in cross-program coordination of administrative and operational activities simply because 
their program is just getting started: funding awards were made to districts as recently as September 
1998. TLCF managers said that they have simply not had time to do any program administration 
beyond awarding grants, but have every intention of coordinating administrative and operational 
activities with other programs in the future: 



The newness of my program and other federal technology initiatives makes it difficult to 
coordinate more. Technology leadership is new and very busy. We haven’t had time to 
coordinate with everyone yet. 

We’re a relatively new program. We’re just getting rolling. In year two, we started a much 
fuller collaboration and we intend to expand [it] in year three. 



Regarding the reasons why the smaller and discretionary grants programs— Even Start, Migrant 
Education, and Neglected or Delinquent— are not participating in cross-program coordination of 
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administrative and operational activities, many managers of the formula-based programs explained that, 
at a practical level, the timelines and deadlines for applications for most discretionary grants programs 
simply do not match those for formula-based programs, making coordination difficult: 

Because of the funding cycle of the programs, the times at which the local applications for 
competitive grants are due are staggered and local applicants aren’t applying for more than one 
competitive grant at a time. (Title I) 

We decided to consolidate activities and programs with the ones that were most feasible and we 
just haven’t expanded yet. These programs (Titles I, II, IV, and VI) work together the best. . . . 
Migrant only operates during the summer, Even Start just wants to do everything themselves. 

It just worked to use these programs; they have the same clients, reporting periods, etc. (Title I) 

We started the consolidated process in what we thought would be a small, logical way. The 
four programs included (Titles I, II, IV, and VI) have lots of commonalities related to 
administration and operations. We may add other programs as we get further along; we have 
to start somewhere and didn’t want a large group to begin with. (Eisenhower Professional 
Development) 



The reasons that smaller and discretionary grants program administrators offered for not 
participating in cross-program coordination of administrative and operational activities were fairly 
specific. For example, several coordinators of Migrant Education explained that because the migrant 
program operated in the summer months, there was no opportunity to coordinate services and activities 
with other programs, as one Migrant Education administrator’s comment conveys: “We don’t 
coordinate with any other programs because the migrant program is in the summer and other programs 
operate during the school year.” 



As was true in 1996-97, some of these programs’ administrators expressed concerns that 
coordination of administrative and operational activities creates the opportunity for smaller programs to 
be absorbed by larger ones and that the needs of the populations served by the smaller programs could 
go unmet: 

From the perspective of a state that operates its Migrant Program almost exclusively in the 
summer, all of the emphasis on coordinating and consolidating planning for the regular school 
year isn’t a problem, but it puts Migrant to the side. It’s not a huge barrier, but it’s easy for 
states to leave mobile kids out. It just requires that I be vigilant that Migrant funding be used 
to meet [the] needs of migrant kids before funds go to overall school improvement. (Migrant 
Education) 



A few administrators of some of these smaller programs, nevertheless, recognized the potential benefits 
of cross-program coordination to the particular needs of the populations they serve: I like that I can 

get other team members involved in Even Start. They are now more knowledgeable of Even Start than 



before. On site visits, as a team, this enables others to make more informed and broader assessments 
of the program: more people can comment on what they see. The consolidation and coordination is 
hard, but in the long run, it will be good for the [SEA] and the IASA programs.” (Even Start) 

Program focus. Several administrators explained that they did not coordinate with programs 
that did not share the same program purposes and focus; this shows incomplete acceptance of the notion 
behind consolidated planning, which was to enable states “to plan how to use all of their federal funds 
to support overall state goals” (ED, 1996a; p. 7). The following administrators’ comments suggest that 
there are still significant numbers of state managers who were continuing to focus on program-specific 
concerns rather than on broader issues related to standards-based reform and improving student 
achievement: 

It depends on the particular activity. If interests and objectives align, we coordinate. (Safe and 
Drug-Free Schools and Communities) 

Similarities in focus among the programs are the reason I coordinate with some programs and 
not with others. (TLCF) 

The reason for coordinating with some programs and not with others is the focus of those 
programs. (Title VI) 

We only coordinate with programs that focus on families with young children. (Even Start) 

I’m more likely to coordinate with programs that have direct implications for Migrant 
Education. (Migrant Education) 



Consolidated Administrative Funding 

Each federal program allows state education agencies to set aside a small percentage of the 
funds for state-level program administration. These set-asides typically pay the salaries of state 
coordinators as well as covering the other expenses of application review, technical assistance, 
monitoring, and the like. The IASA law allowed states to make a change in the way they accounted for 
these funds: it authorized them to consolidate into a single pool the administrative set-asides under 
Title I, Even Start, Migrant Education, Neglected or Delinquent, Eisenhower Professional 
Development, TLCF, Safe and Drug-Free Schools and Communities, Title VI, and Goals 2000. The 
notion behind this additional flexibility was that it would “make it easier [for state administrators] to 
plan across programs” (ED, 1996a; p. 7). 

Not every state was eligible for this consolidation; the law provided this option only for SEAs 
in which the majority of the agency’s resources came from non-federal sources. As was true in 1996- 
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97, among those states that were eligible in 1998, it appears that some did not choose to consolidate 
funds and some were selective about including programs in the consolidation. 

Extent to which state administrators experienced a consolidation of administrative funds. In 
just five states did state administrators of the eligible programs report an across-the-board consolidation 
of state administrative funds; in another three, none answered “yes” to this question (i.e., all 
administrators said either “no” or “don’t know”). Aside from these 8 states where the picture was 
clear, an additional 10 states had just one administrator whose report differed from that of his or her 
colleagues (e.g., there were eight “no” answers and one “yes,” or eight “yes” answers and one no 
or “don’t know”). Compared to data collected in 1996-97, there were fewer states (18 compared to 
28) in which most administrators were sure that their state is or is not consolidating administrative 
funds. 



The remaining 33 states presented a murkier picture, with two or more state administrators 
disagreeing with their colleagues; a typical pattern of responses in these states was two saying “yes,” 
four “no,” and one “don’t know.” As was true in 1996-97, these were probably states in which 
consolidation had been put in place on a limited basis, across just a few of these nine programs. 

Among those states where half the eligible programs were consolidating and the other half were not, 
there were no apparent patterns in terms of the types of programs that tended to consolidate and those 
that did not, with one exception: Goals 2000 and TLCF were least likely to be part of the 
consolidation. Slightly more administrators in 1998 (53 or 12 percent) than in 1996-97 (39 or 11 
percent) reported that they did not know whether their state had consolidated its administrative funding; 
of these, TLCF administrators (the newest program and the one least likely to be included in 
coordinated activities) were the most numerous. 

The rather dramatic decrease in the number of states reporting that they have consolidated 
administrative funds since 1996-97 may be explained, in part, by the fact that many SEAs experienced 
significant turnover in the past few years; consequently, some program managers are fairly new to their 
positions and may simply not know whether their program’s administrative funds are part of a 
consolidation. Another possibility is that state administrators in 1998 have a better understanding of the 
legislation and of administrative operations within their SEA and are able to more accurately respond to 
the question about whether their state is consolidating its administrative funding than they were in 1996- 
97. 



It appears that among the nine states that reported an across-the-board consolidation of state 
administrative funds in 1996-97, six are continuing, for the most part, to consolidate (although no 
longer an across-the-board consolidation, a significant majority of administrators in these states 
reported consolidation). Among the remaining three states, one state had experienced significant staff 
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turnover and two of the nine respondents in that state did not know whether their state was 
consolidating administrative funds. In the other two states, there appeared to be a breaking in the 
ranks. That is, some administrators (two in each state; all four administer small federal programs) have 
apparently decided to discontinue their participation in the consolidation of administrative funds within 
the SEA. 



In addition, a significant majority of administrators in six states that did not report consolidation 
in the baseline study now report consolidating their administrative funds. In 1996-97, the majority of 
administrators in three of these states had said their state was not consolidating funds; in 1998, a 
majority of administrators in two of these three states indicated that their SEA had been downsized, 
which may have prompted the need to organize and use resources more efficiently. As one 
administrator explained: “With diminishing dollars at the state level, the SEA is dipping further into 
areas where federal funds can complement state funds.” (Migrant Education) 

Among the eight states that definitively reported not consolidating administrative funds in 1996- 
97, six continued in 1998 to forgo the use of this particular legislative provision. Of the remaining two 
states, one has a majority of administrators participating in the consolidation; the other has Title I and 
Even Start now making use of this provision. Some of the reasons administrators offered for why their 
states are not consolidating administrative funds were that: (1) their state legislature prohibits funds 
consolidation and (2) budget offices within SEAs— fearing audit exceptions— are not prepared to handle 
funds consolidation. 



Effects of consolidating administrative funds. To get a sense of the effects of consolidation, 
we can look at two groups of respondents: those in the 13 SEAs in which a significant majority of 
state administrators answered “yes” to the consolidation question; and all respondents who answered 
“yes,” regardless of what their colleagues said. 

Among the 13 states where a significant majority of state administrators said funds had been 
consolidated, there were five states in which most respondents said their work had been affected at least 
“somewhat” or “to a considerable extent” by administrative funds consolidation. Among these five 
states, program managers were very clear about whether the effects of consolidation were positive or 
negative; perceptions of effects varied by state rather than by program. A majority of administrators in 
four of these five states saw the effects of consolidation as positive. The following finding was typical: 

• The consolidation has changed the way the SEA functions to a considerable extent, 
making staff responsible for regions and specific content areas rather than specific 
programs. The Title I director said that it has increased the SEA’s capacity to engage 
in a variety of reform activities, including the provision of technical assistance. 
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In the state where a majority of administrators saw the effects of consolidation as negative, the 
following comments were typical: 

The consolidation reduces the available resources to administer the program. (Eisenhower 
Professional Development) 

I think the biggest effect is that it has definitely limited the resources available for administering 
the programs. The dollars are not readily available. What we are given each year depends on 
the department. (Even Start) 

We lost about one-third of our money, which tremendously impacts our ability to have 
sufficient staff and to use discretionary funds. (Safe and Drug-Free Schools and Communities) 

Finally, in one state where all state administrators said funds had been consolidated, most of 
them said that funds consolidation had not affected their own work at all. This response seemed to 
reflect a combination of perceptions: that they had already worked with one another before the 
reauthorization and that the change had not affected the way they worked with local districts. 



Looking at all the state administrators across all states who said their agency had consolidated 
administrative funds, 69 percent reported some effect on their own work (Table 4), with no significant 
variation by program. While a few administrators cited, as they had in 1996-97, the lifting of a 
requirement to keep "time and effort" logs as a positive effect of funds consolidation, the tone of most 
comments in 1998 regarding the effects of funds consolidation was decidedly negative. Some 
complained that consolidation had resulted in their having less money to administer their programs and 
to award to subgrantees: “I now have less program money at the state level as a result of the 
consolidation” (Eisenhower Professional Development) and “Less money goes to the subgrantee 
because funds get stuck in the SEA” (Safe and Drug-Free Schools and Communities). Other 
administrators complained that they had lost control of their program resources as a result of 
consolidation: 

I have less control. I now have to ask permission to earmark money for training. I used to be 
able to make my own decisions for spending money. Money is now earmarked by the powers 
that be. (Safe and Drug-Free Schools and Communities) 

Consolidation of administrative funds has in some ways led state and local programs to become 
nonexistent by shifting the focus away from more targeted funding. It has limited us as to how 
much we can do with the administration of the program because we don’t have access to those 
funds. It has allowed other policymakers to determine what happens in the Eisenhower 
Professional Development program, even if they don’t have a true sense of the needs in science 
and mathematics professional development. (Eisenhower Professional Development) 
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Table 4 



Extent to Which Consolidation of A dminis trative Funding 
Affects the Way Administrators Do Their Job, by Program 

(N = 183) 



To what extent has this consolidation of funds affected the way you do your job? 






Federally Funded 
Education Programs 


Number of State Administrators Reporting i 

That Consolidation of Funds Has Affected Their Job: 

To a 

Considerable 

Extent Somewhat Not at All Don't Know 


Title I, Part A (N= 27) 


11 


10 


5 


1 


Safe & Drug Free Schools and 


Communities (N = 2 1 ) 


6 


9 


6 


0 


Goals 2000 (N = 13) 


5 


4 


3 


1 


Technology Literacy 


Challenge Fund (N=8) 


1 


4 


1 


1 


Title VI (N=23) 


7 


6 


9 


2 


Eisenhower Professional 


Development (N=26) 


12 


8 


5 


1 


Education of Migratory 


Children (N=25) 


9 


8 


7 


1 


Even Start Family Literacy (N= 19) 


10 


5 


4 


0 


Programs for Neglected or Delinquent I 


Children and Youth (N=21) 


7 


5 


8 


1 


TOTAL 


68 


59 


48 


8 
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Despite administrators’ generally negative views of the effects of funds consolidation on their 
work, a substantial majority of administrators for Title I, Title VI, and Eisenhower Professional 
Development reported that consolidation had increased their programs’ capacity to: (1) participate in 
state-level cross-program planning, (2) coordinate program services and operations among other state 
and federal programs, and (3) accept and review consolidated subgrant applications (Table 5). The 
majority of Title I administrators, in particular, believed that consolidating administrative funding 
increased their capacity to do a range of things, including providing technical assistance to subgrantees, 
monitoring local projects, and participating in the development of state standards and assessments. The 
only activity that fewer than half the Title I administrators believed had been positively affected by 
consolidating administrative funds was engaging in data-driven decision-making. Altogether, no other 
program gave consolidation of administrative funding such a ringing endorsement. 

The apparent lack of a connection between general negative attitudes regarding funds 
consolidation and the acknowledgment that consolidation has increased some programs’ administrative 
and operational capacity may be explained in terms of the administrative functions that state managers 
value. That is, although consolidation of administrative funds increases administrators’ capacity to do 
some things, they are not the types of administrative or operational activities or tasks that 
administrators necessarily want to do or for which they require additional capacity. 

Those programs for which a majority of administrators did not perceive an increase in their 
capacity to engage in and coordinate broader administrative tasks as a result of consolidating 
administrative funds included Safe and Drug-Free Schools and Communities, TLCF, Migrant 
Education, Even Start, and Neglected or Delinquent. With the exception of Safe and Drug-Free 
Schools and Communities, many of these programs were not generally engaging in coordinated 
program activities and therefore would not have recognized or benefited from the increased capacity 
that consolidating administrative funding seemingly afforded administrators of Titles I and VI, and 
Eisenhower Professional Development. Interestingly, administrators of Migrant Education and 
Neglected or Delinquent did not perceive effects from the consolidation of funds in 1996-97, either. As 
was true then, because Migrant Education and Neglected or Delinquent programs typically serve 
children outside the regular school setting, they may offer more limited opportunities to share 
administrative responsibilities such as, for example, monitoring. Many Migrant Education programs 
operate during the summer months, thus limiting the opportunities to participate in integrated 
monitoring visits during the regular school year. Similarly, Neglected or Delinquent programs 
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Table 5 



State Administrators Reporting That Consolidating 
Administrative Funds Has INCREASED Their Program’s 
Capacity to Do Things, by Program 
(N = 183) 



To what extent has consolidating administrative funds affected your program’s capacity to do the following? 



Number of State Administrators Reporting That Consolidating Administrative Funds 
Has INCREASED Their Program’s Capacity to: 



Federally Funded 


Provide 

Technical 

Assistance 

to 


Participate 
in State- 
Level Cross- 
Program 


Coordinate 
Program 
Services and 
Operations 
Among 
Other State 
and Federal 


Monitor 

Local 


Participate in 
the 

Development 
of State 


Participate in 
the 

Development 
of State 


Accept and 
Review 
Consolidated 
Subgrant 


Engage 
in Data- 
Driven 
Decision- 




Education Programs 


Subgrantees 


Planning 


Programs 


Projects 


Standards 


Assessments 


Applications 


making 


Other 


Title I, Part A (N=27) 


17 


22 


22 


17 


15 


16 


18 


12 


1 


Safe & Drug Free Schools 


and Communities (N=21) 


8 


9 


9 


5 


5 


3 


8 


3 


1 


Goals 2000 (N = 13) 


7 


10 


9 


6 


7 


5 


6 


8 


0 


Technology Literacy 


Challenge Fund (N = 8) 


3 


5 


5 


5 


3 


3 


3 


2 


0 


Title VI (N=23) 


8 


18 


16 


10 


9 


8 


14 


7 


0 


Eisenhower Professional 


Development (N =26) 


13 


20 


19 


13 


10 


9 


15 


7 


0 


Education of Migratory 


Children (N=25) 


9 


12 


11 


6 


6 


6 


10 


7 


0 


Even Start Family 


Literacy (N = 19) 


7 


10 


10 


9 


6 


3 


7 


7 


0 


Programs for Neglected or 
Delinquent Children and 
Youth (N=21) 


12 


10 


11 


8 


6 


6 


10 


8 


0 


TOTAL 


84 


116 


112 


79 


67 


59 


91 


61 


2 
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(Subpart l) 4 operate in state institutions, where other state administrators of federal education programs 
are unlikely to visit as part of their monitoring. 



State Participation in the Education Flexibility Partnership Demonstration Program or “Ed-Flex” 

Established by the Goals 2000: Educate America Act, Ed-Flex gives participating states the 
“power to waive requirements of certain federal education programs, including Title I and the 
Eisenhower Professional Development Programs” in exchange for increased accountability for results. 
Without Ed-Flex, states and school districts may ask the Secretary of Education to waive these 
requirements; with Ed-Flex, states have the authority to make those decisions at the state level (Ed-Flex 
Fact Sheet, ED, 1996b). 

Among the 12 states that received Ed-Flex authority, administrators reported that the local 
waiver provision was, at best, underutilized and at worst, ignored. Administrators in nine of the 12 
states said that they believed that their state and districts were not fully using the Ed-Flex authority. 
Specifically, of the 10 administrators who were able to estimate the number of districts in their states 
that had received waivers, seven said the number was 20 or fewer. The most common explanation 
administrators offered for underuse of the waiver authority was that the reauthorized legislation grants 
programs sufficient flexibility, thereby rendering Ed-Flex superfluous; as one administrator pointed 
out: “There is so much flexibility built into ESEA that there is not a big need for more.” In 
characterizing the way districts have used the waiver authority, many state administrators lamented the 
fact that few districts seem to recognize their state’s Ed-Flex status as an opportunity for change. 
Rather, districts are asking for waivers of some of their more mundane administrative responsibilities— 
such as keeping time and effort logs— and the consequent effects on program administration and 
operations are negligible: 

I feel the Ed-Flex is underutilized. No major innovative waivers are asked for. It hasn’t 
increased or decreased the performance of the program, although the field does seem happier. 

As an Ed-Flex state, we had the ability to give administrative waivers, so we waived time and 
effort requirements in some districts— but that didn’t affect anything we did except to reduce 
administrative record keeping. 

We have a real dichotomy here. We have a lot of flexibility, but LEAs haven’t taken 
advantage of it. Most LEAs use the flexibility to implement Title I schoolwide programs. 



4 Subpart 2 of the Neglected or Delinquent Program is covered by another ED-funded study and was therefore 
not included in either the baseline or follow-up studies of state implementation of ESEA or Goals 2000. 



What happens is a lot of times with the federal laws, what people do is they assume that what 

they used the funds for are the only things they can use them for and don’t think “outside the 
box.” 



On a more positive note, most state administrators (8) reported including student performance 
among their criteria for assessing the impact of waivers and said they will revoke district waivers as a 
result of poor student performance — although none had, as yet, done so. 

We look for the impact of the waiver on student performance as well as program 
performance— what they’ve been able to accomplish as a result of the waiver and how it has 
affected the performance of students. 

When districts apply for waivers, they have to provide us with information on how the waiver 
they are getting is going to increase student performance, and how they are going to measure 
it. We ask them then to report their progress in reports, using the criteria that they have 
established. 



Finally, despite their rather limited use among districts, few state administrators would be < 
willing to discard waivers as meaningless: most (8) say they believe-to some or to a great extent— that 
the Ed-Flex waiver authority (1) furthers state reform efforts and (2) improves the coordination of 
federal resources with state reform efforts. 



Local Administrative Flexibility 

Goals 2000 and IASA amendments aimed to increase flexibility in schools and school districts 
so that administrative requirements would not impede progress in bringing all students to high 
standards. The survey asked state administrators to report on the extent to which the administrative 
flexibility afforded state education agencies was being extended to the local level. 



Requiring or Accepting Local Consolidated Plans 

Slightly more than half the state administrators responding to the survey said they allowed 
school districts or other subgrantees to submit a single consolidated application-a plan describing the 
intended uses of funds under more than one program. That is, about 57 percent of all program 
administrators reported that they required (30 percent) or accepted (27 percent) consolidated 
applications from districts (Table 6). As usual, however, there were important program-by-program 
variations to this finding. As was true in 1996-97, more than three-fourths of the administrators for the 
formula grants programs- Title I, Safe and Drug-Free Schools and Communities, Eisenhower 
Professional Development, and Title Vi-reported that they required or accepted consolidated 
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Table 6 



Policy on Consolidated District 
Applications, by Program 
(N=434) 



Does your program require or accept consolidated plans or applications (that is, combined applications for more 
than one program) from local school districts or other subgrantees? 



Number of State Administrators Reporting That Their Programs: 





Requires 


Accepts 


Requires a 


Piloted 

Consolidated 

Subgrant 

Applications 


Makes Other 




Consolidated 


Consolidated 


Separate 


in Some 


Arrangements 


Federally Funded 


Subgrant 


Subgrant 


Sub grant 


Districts in 


for Subgrant 


Education Programs 


Applications 


Applications 


Application 


1997-98 


Applications 



Title I, Part A (N-49) 


25 


16 


5 


2 


1 


Safe & Drug Free Schools and 
Communities (N=48) 


21 


15 


9 


1 


2 


Goals 2000 (N=46) 


6 


5 


35 


0 


0 


Technology Literacy Challenge 
Fund (N=49) 


3 


6 


37 


1 


2 


Title VI (N-48) 


22 


20 


6 


0 


0 


Eisenhower Professional 
Development (N=50) 


22 


20 


6 


1 


1 


Education of Migratory 
Children (N-47) 


11 


16 


16 


1 


3 


Even Start Family Literacy (N = 50) 


6 


6 


38 


0 


0 


Programs for Neglected or Delinquent 
Children and Youth (N=47) 


14 


13 


18 


1 


1 


TOTAL 


130 


117 


170 


7 


10 
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applications, as compared with fewer than one-third of the administrators for the discretionary grants 
programs, including Goals 2000, TLCF, and Even Start. 

Consolidated applications or plans were most frequently in place, then, across the programs 
that offered formula funding to districts. In explaining this phenomenon, many respondents noted— as 
they had in 1996-97— that accepting or requiring local consolidated applications was a new 
administrative procedure that was simpler to introduce to a core group of programs— especially ones 
that were not targeting particular populations— to test the processes of developing a common subgrant 
application and reviewing consolidated applications. Administrators explained in 1996-97 that once 
they had an opportunity to learn from their experience, they might eventually include more federally 
funded programs in a consolidated process. Since 1996-97, however, it appears that some states have 
tried introducing a few of the discretionary grants programs into local consolidated plans, only to 
quickly reject the option because programs (largely the discretionary grants programs themselves) were 
displeased with the quality and quantity of information the applications provided: 

I have talked with folks who have done these consolidated things, and we have enough 
problems trying to find the information we need without sifting through everyone else’s 
information. I would rather have the things I’m looking for, rather than things the Eisenhower 
Professional Development and Title VI person would need, but that I don’t need. It is the 
efficiency of getting things approved and getting on with it. (Migrant/Neglected or Delinquent) 

Another important development in state-level program administration since 1996-97 is the 
increase in the percentage of administrators who reported requiring— versus merely accepting— 
consolidated local plans or applications. In 1998, 30 percent of program administrators required local 
consolidated applications, compared to 22 percent who did so in 1996-97. This increase, again, signals 
a shift in the mindset of state administrators away from state-level administrative concerns and toward 
finding ways to better support local efforts to improve student achievement, as the following comments 
illustrate: 



Consolidated local applications are required in order to coordinate the utilization and impact of 
funds toward all students’ attainment of challenging performance standards. (Title I) 

[We require consolidated local applications] because we are trying to promote whole-school 
planning and the consolidated use of resources, rather than separate planning by program 
streams. We want districts to look at the needs of students and how the various streams can 
promote meeting them. We want districts to look at what the kids need, and then the resources 
available to meet the needs, rather than [engaging in] separate planning by program stream 
before examining student needs. (Goals 2000) 

The SEA wants school districts to see, at a glance, all of the different funding sources that are 
available to them. They also want to allow districts to coordinate resources and programs 
better— to know what the right hand and the left hand are doing. (Even Start) 



Quality of local consolidated plans. The majority of program administrators who accept or 
require local consolidated plans or applications expressed general satisfaction with the quality of certain 
information that the local plans provide. More than half the administrators who require or accept local 
consolidated plans or applications reported that they provide information “very well” or “fairly well” 
on summaries of services rendered (58 percent) and indicators of collaboration with other programs (68 
percent) (Table 7a). Important program-by-program variations to these findings were present, 
however. That is, two-thirds or more of the administrators of Title I, Goals 2000, and Title VI 
reported that consolidated local plans or applications provided summaries of services rendered very 
well or fairly well, as compared to fewer than half the administrators for Safe and Drug-Free Schools 
and Communities, TLCF, and the Migrant Program. Regarding information being provided “not very 
well” or “poorly,” 30 percent of respondents said that the local consolidated plans or applications 
provided “evidence of additional services needed” not very well or poorly, with Title I, Safe and Drug- 
Free Schools and Communities, and Eisenhower Professional Development among the programs most 
frequently selecting this response option (Table 7b). 

Local implementation of consolidated plans. Regarding the implementation of consolidated 
local plans, more than one-fourth of the 264 state administrators who reported requiring or accepting 
consolidated local plans said that they did not have enough information about local implementation to 
report on its strengths and weaknesses. Among those who did believe they had enough information to 
comment (190 state administrators), the vast majority focused on the strengths of local implementation, 
with 83 percent reporting that working together across programs has “helped local staff learn about 
each other’s programs and identify opportunities to coordinate administrative and operational 
activities.” In addition, 71 percent believed that a consolidated plan sharpens local educators’ focus on 
enabling all students to meet the standards; this was a particularly frequent selection among 
administrators of Title VI, Eisenhower Professional Development, and Goals 2000 (Table 8). 



Requiring Separate Subgrant Plans or Applications 

Administrators whose programs required them to make discretionary grant awards were more 
frequently reporting requiring separate subgrant applications than administrators of other programs. 

For example, Even Start, Goals 2000, and TLCF, all of which target limited resources to districts most 
in need, were less likely than other programs to require or accept consolidated applications. 5 When 
asked why programs continue to require separate local applications, the most common explanation was 

5 For Even Start, this may also be a result of the fact that eligible entities for subgrant awards are not simply 
LEAs, but LEAs in partnership with at least one other entity, such as a non-profit, community-based organization. 
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State Administrators Reporting on the Strengths and Weaknesses 
of Local Implementation of Consolidated Plans, by Program 

(N=264) 
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that competitive grants programs operate on different grant cycles from the formula-based programs. 
Other explanations focused on (1) adherence to program requirements that, in some administrators’ 
opinions, dictated separate applications, and (2) the notion that consolidated applications could not 
provide program administrators with the information they needed to judge program quality or local 
need or that other program administrators would be reviewing the applications but would not have the 
expertise to make those judgments. The following comments were typical: 

Separate applications are important for flexibility, mobility, and ability to respond to the needs 
of LEAs quickly. (Migrant Education) 

The information required by the N or D program is so different from others and because there 
are so few N or D programs, it makes it difficult to operate through a consolidated application. 
(Neglected or Delinquent) 

There is the concern that the other programs would not have the expertise to evaluate what 
would be an appropriate use of technology and what would not. (TLCF) 



Factors Influencing Subgrantees’ Decision to Submit a Consolidated Plan 

Interviews with local federal program administrators about factors that influenced their decision 
regarding whether to submit to the state a consolidated application suggested that some states were not 
making it easy for districts to submit such plans. This finding represents a different perspective than 
that offered by state administrators in 1996-97, when several state administrators reported that many 
local administrators did not want to submit consolidated plans because they were still working within 
the boundaries of their categorical programs and resisted efforts to coordinate program services and 
activities. (Interviews with local administrators were not conducted as part of the baseline study.) In 
1998, during site visits to 16 districts (two in each of eight states), we interviewed local federal 
program administrators and asked them about the factors that influenced their decision regarding 
whether to submit a consolidated application. Many explained that their state was simply making it too 
difficult to design a consolidated application: 



The state is requiring too much in the application. They are really asking for a separate 
application for each program stapled together. Districts do not have to mix and mingle abilities 
with their funds that make applying worth it. That is why there are only a couple consolidated 
districts in the state, and they are very small and rural. 

Although state administrators may say that they accept consolidated local applications, most do 
not actively encourage it and none provide technical assistance to districts on how to go about 
writing such an application. In addition, because the state has not removed its requirement for 
time and effort logs, writing a consolidated application is not a shortcut. 
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Some local administrators suggested that state managers do not know what a local consolidated plan 
ought to look like— nor how they can support local program coordination— and therefore are unable to 
offer districts much assistance. As one local administrator explained: “The state only encourages 
collaboration; they don’t help locals consolidate. All the programs have separate accountability 
requirements and do not share mutual objectives.” Another administrator pointed out that his district 
had received their program approval letters much sooner when they submitted separate applications: 
“We submitted our consolidated application in July 1998 and didn’t receive formal approval until 
November 1998; in the old days, we would have gotten our approval letters by September. Once the 
state receives our application, they break it up by program for review.” 

On the other hand, in districts where consolidated planning was thought to be supported by the 
state, local administrators spoke glowingly about the opportunities consolidation was affording their 
communities. Specifically, local administrators in two states talked about consolidation opening the 
doors to a variety of new funding sources: 

There was an all-day meeting for Title I coordinators years ago related to the importance and 
inevitability of consolidating. We consolidated the first time we were able. They were good at 
giving us information and clear that they didn’t know how it would work. The state is 
committed to the idea of consolidated planning. In less than four years, this school district has 
gained over $2.5 million in competitive grants. That’s because the state could see we were 
trying to do consolidated planning and had our strategic plan. 

Having a consolidated plan gives us leverage to go after additional funding; we’ve gotten 
national grant awards from NSF and General Electric. These come as a result of presenting a 
unified local plan for improvement. 

Summary: The Status of “Flexibility” 

Compared to survey data collected in 1996-97, current findings show that perceptions of 
flexibility have changed significantly. Two-thirds of state administrators surveyed in 1998 believed that 
their flexibility had increased since reauthorization, whereas in 1996-97, Title I was the only program 
in which a majority of administrators reported an increase in flexibility. 

Program managers’ descriptions of the flexibility available to them also signaled a shift in 
perceptions since the baseline study. In 1998, many program managers described efforts to help 
districts find ways to use federal program resources to meet local needs; survey responses in 1996-97 
indicated that program administrators were thinking of flexibility in a more limited way— primarily in 
terms of opportunities to coordinate and collaborate at the state level. 
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In another positive development since 1996-97, the study found an increase in the percentage of 
administrators who reported requiring— versus merely accepting— consolidated local plans or 
applications. This increase, again, signals a shift among state administrators, away from state-level 
administrative concerns and toward finding ways to better support and encourage local reform efforts. 

However, the study also found some areas in which implementation of flexibility provisions fell 
short of the original hopes for the reauthorized legislation. Survey results showed that fewer states 
reported an across-the-board consolidation of administrative funds than in 1996-97, and the tone of 
most comments regarding the effects of funds consolidation was decidedly negative. Findings also 
suggested that the Ed-Flex waiver authority was underutilized. Finally, as in 1996-97, few 
administrators mentioned the relationship between flexibility and improvements in program services or 
student outcomes. 



III. Accountability: To What Extent Are States Implementing 
Standards-Based Accountability Systems? 



IASA, the federal legislation amending ESEA, includes provisions that connect program 
accountability to state systems of challenging content and performance standards. According to ED’s 
cross-cutting guidance, the idea behind this mandate was to “[1] improve coordination of federal 
programs with state reforms and [2] instill in federal programs a culture of accountability and continual 
improvement” (ED, 1996a; p. 9). Advocates hoped that the accountability mechanisms created in 
response to these amendments would direct educators’ and program administrators’ attention to the 
challenge of bringing all students to high standards— and, further, that the magnitude of this challenge 
could stimulate big changes in program services. For the legislation to fulfill this hope, however, 
several structures and processes would have to be in place. Not only would states need ways of 
measuring student achievement, they would also need to lead districts and schools in the use of data to 
stimulate and guide improvements in program services. 

In 1996-97, we found that with the exception of following some mandated procedures, such as 
identifying Title I schools in need of improvement, most program administrators gave rather limited 
evidence that they were attending to student performance. The follow-up survey explored the extent to 
which, four years after reauthorization, state administrators were organizing their work around aligning 
program services and operations with state content and student performance standards in an effort to 
improve student achievement. In addition, it explored the extent to which state administrators were 
using a variety of accountability tools— including student performance data, program implementation 
data, and program monitoring systems— to press vigorously for improving student performance. In 
addition, the follow-up study explored the extent to which Regional Service Teams— the federal 
program monitoring system— also worked to support state and local reform in an effort to improve 
student achievement. 



Content Standards: How Are They Affecting State Administration of 
Federally Funded Programs? 

After four years of implementing the law, it appears that state program administrators have 
come to recognize that supporting standards-based reform is part of their programs’ purpose. 
Moreover, many administrators in their responses to open-ended survey questions mentioned raising 
student achievement— the ultimate goal of standards-based reform— as among their more fundamental 
program goals. 
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Content Standards and Federal Program Administration 



In all states, content standards were part of the vocabulary of a significant number of state 
officials who manage federal programs. Overall, it appears that the majority of state administrators 
recognize that supporting standards-based reform is part of their programs’ purpose: few respondents 
(6 percent) said that standards are “not relevant” to the services their program provides or that the 
program staff do not have the time (14 percent) or the expertise (4 percent) “to communicate a new 
program purpose driven by state standards” (Table 9). Evidence of attention to standards comes to 
light in administrators’ descriptions of the changes they have made to some of their regular 
administrative tasks, including designing and approving local applications for federal funding and 
providing technical assistance. Because these comments were offered in response to open-ended 
questions, we cannot say exactly how many program administrators share these views, but they arose in 
more than a few interviews. 

Local applications for federal funding. In responses to a variety of open-ended questions 
intended to shed light on the ways in which program administration and operations may have changed 
in the years since the reauthorization of ESEA, many state administrators— of all nine programs — 
referred to changes they have made in their application requirements in an attempt to focus local 
activity on supporting standards-based education, as the following comments illustrate: 

Standards and assessments are now the total focus of everything we do with our funds. They 
are part of the consolidated application— districts need to look at needs, and needs are based on 
what it will take to get the state standards in place. (Eisenhower Professional Development) 

In the next RFP for district improvement plans, we are asking people to develop plans that 
show how they are aligning local curriculum and assessment with state assessments and 
standards and also how they are using their data from [the state assessment] to realign their 
local practices and procedures. (Goals 2000) 

The RFP was changed to include a requirement to demonstrate how the program will support 
both the National Education Goals and the state standards. (Even Start) 

Technical assistance. Many state administrators (although, by no means all) mentioned that 
attention to standards has helped bring focus to— and thereby improve— the quality of the technical 
assistance they fund or provide to districts: 

For technical assistance, we focus on how Title I is based on standards and an integral part of 
the schools. We are looking at more global issues, not just Title I issues. We are no longer in 
our silos. (Title I) 
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State Administrators Who Report the Following Problems 
With the Implementation of Standards-Based Reform, by Program 

(N=430) 
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Because of standards and assessments, there’s been more of a focus. We’re providing technical 
assistance to districts in and around Eisenhower Professional Development. We provided an 
overview of the test, then we moved into curriculum workshops to do training. Then districts 
asked us how to use the rubrics, how to fold them into instruction, how to use them with 
students. We provided that kind of technical assistance, and they used their funds to continue 
that kind of training. (Eisenhower Professional Development and Title VI) 

Although standards seem to be taking a firmer hold on state administrators’ conceptions of their 
program purposes and objectives, standards are not the sole anchor for federal program efforts. The 
survey data provide a window on administrators’ attention to standards as a focus for capacity building: 
all program administrators were asked about the topics on which they provide or fund technical 
assistance to grantees. As was true in 1996-97, the state administrators of Eisenhower Professional 
Development, Title I, and Goals 2000 substantially outnumbered their counterparts in other programs 
in choosing “content or performance standards” as a focus for technical assistance (Table 10). The 
only change in 1998 was the increase in the number of Title VI administrators reporting that they fund 
or provide technical assistance on content or performance standards (from 30 in 1996-97 to 40 in 
1998). Overall, the percentage of state administrators who reported funding or providing technical 
assistance to subgrantees on content or performance standards did not change much, rising from 56 
percent in 1996-97 to 61 percent in 1998. 



Raising Student Achievement 

In 1996-97, substantial numbers of state administrators— in their responses to a series of open- 
ended questions— gave no evidence that their criteria for program success lay in the realm of student 
achievement. To follow up on this finding and delve a little deeper into its origins, we asked state 
administrators in 1998 to describe the ways in which their program’s organization or operations 
affected the extent to which they could focus on improving student achievement. This time with a more 
focused question, state administrators were more likely to say they were addressing student 
achievement as part of their programs’ purposes. They attributed this largely to (1) efforts to 
coordinate program administration and operations among other federal programs and (2) the 
implementation of state standards and assessment systems, sometimes after long delays. 

Regarding cross-program coordination, there seemed to be a strong belief among some that 
coordinating federal program administration and operations at the state level gives managers greater 
incentive to focus on student achievement. That is, by coordinating technical assistance, monitoring, 
and other administrative tasks— and thereby reducing their administrative burdens— managers believe 
they have more time to focus on the complicated issue of improving student achievement: 
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Table 10 



State Administrators Who Choose Content or 
Performance Standards as a Focus for Capacity Building, by Program 

(N=435) 



In the past year, has your office funded or directly provided technical assistance to subgrantees on [content or performance 
standards]? 


Federally Funded Education Programs 


Number of State Administrators 
Who Fund or Directly Provide 
Subgrantees Technical Assistance on 
Content or Performance Standards 


Number of State Administrators 
Who Do Not Fund or Directly 
Provide Any Technical Assistance to 
Subgrantees 


Title I, Part A (N=49) 


35 


0 


Safe and Drug-Free Schools and 
Communities (N=48) 


19 


0 


Goals 2000 (N=46) 


32 


4 


Technology Literacy Challenge Fund (N = 50) 


25 


2 


Title VI (N=48) 


40 


2 


Eisenhower Professional Development (N = 50) 


46 


0 


Education of Migratory Children (N=47) 


25 


6 


Even Start Family Literacy (N = 50) 


22 


6 


Programs for Neglected or Delinquent Children 
and Youth (N=47) 


23 


14 
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Having to be responsible for all of the programs affects your focus on improving student 
achievement. It has a positive effect because you can help teachers and administrators by 
speaking globally, instead of with a narrow focus, to help them see all of the things they have 
access to in order to make sure that Johnny achieves. (Eisenhower Professional Development) 

Coordination with other programs allows us to focus much more on improving student 
achievement. Encouragement from ED to coordinate access across programs and to make 
“need”— not programs— important, really helps sharpen our focus. (Goals 2000) 

A lack of adequate staff was commonly cited as the reason why programs could not focus on student 
achievement. Indeed, as the following comments illustrate, administrators expressed intense frustration 
about their limited capacity to fulfill administrative tasks, let alone to go beyond minimal requirements: 



Just the fact that we are only a 1 .5 FTE program and there is so much. . . we have to do for 
the feds and stuff, it is pretty hard to focus on anything more than just this program. 
Presumably, kids making healthy choices will ultimately improve their grades, but as far as a 
specific focus on student achievement, we are limited in our capacity to address this issue. 

(Safe and Drug-Free Schools and Communities) 

It’s just me. I have very little staff to provide services in areas that would improve student 
achievement. If I had more staff, I would do a lot more regional training and would really look 
at the needs identified for districts and think about what Eisenhower Professional Development 
can do to help support those needs. (Eisenhower Professional Development) 

Because of understaffing, it’s really hard to dedicate the time we should be dedicating to 
student achievement and improving program coordination. Most of our time gets spent on 
doing things that have to get done. We need more focus on student achievement, and for that, 
we need more staff. (TLCF) 

Regarding the implementation of state standards and assessments, several program managers 
explained that they had not paid particular attention to student achievement in the past simply because it 
did not make sense in the absence of state standards and assessment systems: what, they asked, would 
they be measuring in order to assess student achievement against? As the following comments 
illustrate, having state standards and assessment systems in place can be a fairly powerful lever for 
improving coordination of federal programs with state reforms: 




We’ve got standards and curriculum assessments in place— now we are in the process of 
looking at what the new assessments are telling us and trying to retool the way we teach. We 
are moving to performance testing. We need to get the teachers up to speed on how the test 
will change the way we should be teaching students before talking about sizable gains in student 
performance on tests. We need to lay the groundwork first. (Goals 2000) 

Once the state adopts statewide assessments, we should be better able to measure students’ 
progress and determine the needs of local schools. (Eisenhower Professional Development) 
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Now that the curriculum frameworks and state assessment are in place, the challenge is to help 
students to achieve the state standards. (Title I) 

Finally, although state administrators said more in 1998 about their efforts to focus program 
operations and activities on raising student achievement, they still expressed reluctance to link program 
success to student achievement. As the following section illustrates, student performance data were not 
yet cited as a significant source of information for administrators when judging their program successes 
and problems. This phenomenon, however, may say less about deep-seated convictions or behaviors, 
and more about simple pragmatism. That is, with the exception of Title I, state administrators argued 
that federal program reporting requirements create a disincentive by continuing to focus on process and 
inputs, rather than on outcomes such as improving student performance. 



Uses of Data in SEA Program Management 

As was true in 1996-97, it appeared that the goal of instilling a culture of accountability and 
continual improvement among those administering federally funded programs was unmet. While 
administrators were seemingly collecting or already have at their disposal a great deal of student 
performance and program implementation data, they were not using the data— or, at least, their 
purposes in using the data had little to do with judging the success of federally funded programs in 
raising student achievement. 



Student Performance Data 



Forty-two percent of all state administrators do not require subgrantees to report to the state 
regarding the performance of students participating in their programs. This phenomenon was most 
common among programs that do not provide direct services to students (Eisenhower Professional 
Development, Safe and Drug-Free Schools and Communities, Title VI, and the Technology Literacy 
Challenge Fund). However, substantial numbers of administrators of Migrant Education, Even Start, 
and Neglected or Delinquent also said they do not require their subgrantees to submit student 
performance data. As administrators of these smaller programs point out, however, districts face 
special difficulties in collecting and reporting student performance data for these programs due to the 
circumstances affecting the particular populations they serve. One administrator said: 

With Neglected or Delinquent, it’s the lack of any long-term information on student 
achievement that makes it hard because the kids in N or D are in and out so quickly and their 
information doesn’t go with them. 
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For Migrant Education, the problems are quite similar. Many states offer only summer migrant 
programs, and administrators argued that it is hard to measure student achievement because of migrant 
students’ mobility; in addition, they observed that their program only serves migrant children for two to 
three months out of a year and they do not know what the achievement data would be telling them 
given the limited duration of the intervention. 

As one might expect, given its requirement to identify schools and districts in need of 
improvement, Title I had the highest proportion of administrators (80 percent) who required 
subgrantees to submit student performance data to the state. The types of student performance data that 
administrators most frequently said they required from subgrantees suggest that increased attention to 
student achievement among federal program administrators may be beginning to take hold: of the 250 
administrators who require districts to submit student performance data, 53 percent said they collect 
data on student performance results from the state assessment; and 42 percent said they collect 
performance results from tests other than the state’s assessment (Table 11). Despite these 
advancements, the data show that there is still room for improvement. That is, 19 percent of state 
administrators report that they do not require subgrantees to submit any student performance data to the 
state. 



Unfortunately, however, while administrators may be collecting student performance data, 
survey data also show that the number of state administrators who report using the data for any purpose 
has declined since 1996-97. That is, among the 293 state administrators in 1998 who reported 
requiring student performance data from subgrantees or obtaining such data from other sources, 23 
percent acknowledged that they did not use the data for any purpose, compared to 15 percent in 1996- 
97 (Table 12). This was most common among administrators of Safe and Drug-Free Schools and 
Communities and Migrant Education. Of those who did say they use the student performance data they 
collect, most state administrators said they used it for some combination of purposes. The most 
common purposes were for (1) reporting to the federal government (58 percent); (2) reporting to local 
school districts, schools, and subgrantees (48 percent); (3) identifying districts in need of improvement 
(47 percent); and (4) identifying schools in need of improvement (44 percent). Program by program, 
Title I had the highest frequency of administrators using student performance data for these purposes. 

In fact, Title I administrators were the most avid consumers of student performance data, putting them 
to a variety of uses. That is, Title I administrators reported— in much higher frequencies than did other 
program administrators— that they used student performance data for: (1) identifying districts and 
schools in need of improvement (a legislative requirement); (2) assessing the progress of underserved 
ethnic and demographic groups; (3) reporting to managers in the state agency; and (4) reporting to 
local school districts, schools, and subgrantees. 
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Table 12 

Purposes for Which Program Offices Use Student Performance Data, 
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As was true in 1996-97, although most state administrators reported making some use of data 
on student performance, their answers to other questions showed that such data did not necessarily 
factor into their assessments of overall program quality. When we asked respondents how they judged 
the success of their work, most cited feedback from subgrantees, their own observations, and other 
anecdotal information. Few described their successes and failures in terms of student achievement. The 
following comment was typical: 



Eisenhower Professional Development is supporting standards-based reform, effective models 
of professional development, and the Title I initiatives. It has helped us in having the flexibility 
to address issues that we find important. Also, adding the additional content areas that we can 
focus on was helpful, particularly in the area of reading. (Eisenhower Professional 
Development) 

Again, few state program administrators reported any systematic way of evaluating the success 
of their program; almost none referred to student performance data. A few administrators comments 
highlighted the difficulty and confusion administrators may feel about the prospect of linking program 
success to student achievement: 



It is very difficult to prove a causal relationship between activities funded through the program 
and increased academic achievement — especially because so many reforms are occurring 
simultaneously in schools. (TLCF) 

It’s hard for my program to focus on improvements in student achievement because the students 
move so much between districts and institutions. (Neglected or Delinquent) 

We don’t have a good enough research base on evaluating the effects of technology on student 
performance. We are dealing with policymakers who want to know the bottom line, and the 
best we have is anecdotal evidence, and sometimes that is not good enough. (TLCF) 



Program Implementation Data 

Far more state administrators (95 percent) require subgrantees to submit program 
implementation than student performance data. The types of program implementation data that 
administrators most frequently said they required from subgrantees were: (1) summaries of services 
rendered (85 percent); (2) indicators of collaboration with other programs (63 percent); and (3) 
evidence of varied demographic and ethnic group participation (53 percent) (Table 13). Significant 
program by program variations persisted, however, with Even Start posting the highest percentage of 
administrators requiring almost every type of program implementation data, including summaries of 
services rendered (94 percent); indicators of collaboration with other programs (90 percent), levels of 
parent involvement (88 percent); levels of community involvement (82 percent); evaluations of service 
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Table 13 

Program Implementation Information That Local School Districts, 
Schools, and Subgrantees Are Required to Report to the State Regarding 
Their Federally Funded Program, by Program 
(N=433) 





quality (72 percent); and evidence of additional services needed (60 percent). This may be the result of 
increased program evaluation requirements placed on local Even Start programs, and that more and 
more state Even Start coordinators may have come to value evaluation data and have found the 
resources to fund statewide program evaluations. 

While there was less program by program variation in the numbers of state administrators using 
program implementation versus student performance data, the most common purpose for which these 
data were used was largely the same. That is, most administrators said they use program 
implementation data for reporting to the federal government (74 percent). The second most common 
purpose for which state administrators said they use program implementation data was to identify 
program services that need to be extended or reduced: 60 percent of the 431 state administrators who 
collect program implementation data reported using the data for this purpose (Table 14). In contrast, 
only 41 percent of the 293 administrators who collect student performance data reported using the data 
for this purpose. This variation, again, illustrates the reluctance among most administrators to use 
student performance data in program management. 



Program Effects Resulting from the Availability of Student Performance and Program 
Implementation Data 

The effects resulting from the availability of program implementation and student performance 
data are somewhat disappointing in light of the standards-based reform agenda. That is, with the 
exception of the Title I and Goals 2000 programs, just under half of the state administrators (44 
percent) reported that program implementation and student performance data helped focus program 
staff on student results and achievement. Rather, administrators reported the effects were that such 
data (1) promoted coordination with other federal, state, and local programs (66 percent), and (2) 
improved communication with program subgrantees (64 percent) (Table 15). 



Using the State-Level Procedure that Identifies Failing Schools 

Among the 27 states where it was clear that the state had a process for identifying failing 
schools, a majority of program administrators in 14 states reported that they availed themselves of those 
data and made program decisions based on them. Overall, 118 of the 235 administrators in these 27 
states reported using the state-level data identifying failing schools. Program by program, it appears 
that Title I is the largest consumer of the data, with only one of the 26 administrators responding to the 
survey in these 27 states reporting that such data were not used. Administrators of Safe and Drug-Free 
Schools and Communities, Title VI, Goals 2000, and Neglected or Delinquent were the next most 
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Table 14 

Purposes for Which Program Offices Use Program Implementation Data, 

by Program 
(N=431) 
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Program Effects Resulting from the Availability of 
Program and Student Performance Data, by Program 
(N=429) 





frequent data users, with majorities of administrators of each of these programs reporting using the data 
or being in the process of developing an accountability system that makes use of state-level data 
identifying failing schools (Table 16). 



State Monitoring 



One way for state program offices to maintain a climate of accountability is by monitoring local 
school districts’ programs. Under the new philosophy embodied in the reauthorized ESEA and Goals 
2000, in which specific requirements take a back seat to the pursuit of more effective teaching and 
learning, monitoring could theoretically be retooled to provide a rigorous focus on program 
effectiveness and states’ educational priorities. In 1996-97, although states had made progress in 
dismantling their old systems of monitoring for compliance with program provisions, they had far to go 
in building new monitoring procedures that would send a clear message about a new, standards-based 
accountability framework. In 1998, states were again asked about their monitoring systems. 



Monitoring as Technical Assistance 

Although the focus of monitoring continued to be on program effectiveness rather than 
compliance, 1998 survey data showed that compliance monitoring has not disappeared altogether. 
Rather, it is merely one among many priorities state administrators address through state monitoring 
visits. That is, when asked what priorities are addressed through state monitoring, the vast majority of 
program administrators (89 percent) cited compliance with federal program requirements. Other 
frequently cited priorities included (1) progress in tracking activities outlined in local plans (76 percent) 
and (2) use of federal funds to support state and local reform efforts (69 percent). With the exception 
of Title I, Even Start, and Neglected or Delinquent, only about half the program administrators cited 
student outcomes as a priority addressed through monitoring visits (Table 17). 

Administrators in 1998 were more explicit than they were in 1996-97 about the kinds of 
technical assistance SEAs were offering as part of monitoring. That is, several administrators talked 
about focusing monitoring on program outcomes and finding ways to encourage districts to coordinate 
program services and activities: 

The monitoring is moving more toward standards. It is still in flux. We’re trying to modify 

our [monitoring] items to look more at outcomes and not so much at compliance items. 

(Eisenhower Professional Development) 
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Table 16 



State Administrators Reporting That Their Program Makes Decisions or 
Provides Services Based on State-Level Data Identifying Failing Schools, by Program 

(N=27I) 



Does your program make any decisions or provide any services based on state-level data identifying failing schools? 



Number of State Administrators Reporting That: 



Their Program Is in the 
Process of Developing an 



Federally Funded 
Education Programs 


Their Program Makes 
Decisions or Provides 
Services Based on State- 
level Data Identifying 
Failing Schools 


Accountability System 
That Makes Use of 
State-level Data 
Identifying 
Failing Schools 


Their Program 
Does NOT Make Use of 
State-level Data 
Identifying Failing 
Schools 


They Don’t 
Know 


Title I, Part A (N = 34) 


30 


3 


1 


0 


Safe and Drug-Free 
Schools and Communities 
(N = 33) 


16 


10 


7 


0 


Goals 2000 (N = 30) 


20 


2 


8 


0 


Technology Literacy 
Challenge Fund (N=28) 


13 


7 


8 


0 


Title VI (N = 30) 


16 


5 


9 


0 


Eisenhower Professional 
Development (N=29) 


13 


1 


15 


0 


Education of Migratory 
Children (N=28) 


10 


2 


16 


0 


Even Start Family 
Literacy (N=28) 


11 


2 


15 


0 


Programs for Neglected 
or Delinquent Children 
and Youth (N = 31) 


14 


5 


11 


1 


TOTAL 


143 


37 


90 


1 
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Table 17 



Priorities State Administrators Report Addressing 
Through Monitoring, by Program 
(N=415) 



What priorities are addressed through state monitoring? 



Number of State Administrators Reporting that the Priorities They Address 
Through State Monitoring of Subgrantees Include : 



Federally Funded 
Education Programs 


Student 

Outcomes 


Progress in 
Tracking 
Activities 
Outlined in 
Local Plans 


Title I, Part A (N=48) 


33 


36 


Safe and Drug-Free 1 
Schools and 
Communities (N=46) 


22 


41 


Goals 2000 (N = 43) 


24 


35 


Technology Literacy 
Challenge Fund (N=45) 


21 


38 


Title VI (N=47) 


23 


32 


Eisenhower Professional 
Development (N=46) 


24 


31 


Education of Migratory 
Children (N=45) 


25 


29 


Even Start Family 
Literacy (N=49) 


36 


42 


Programs for Neglected or 
Delinquent Children and 
Youth (N=46) 


29 


.30 


TOTAL 


237 


314 



Use of Federal 
Funds to Support 
State and Local 
Reform Efforts 


Compliance with 
Federal Programs 
Requirements 


Other 


34 


42 


5 


29 


46 


7 


36 


30 


5 


30 


34 


6 


38 


46 


5 


34 


42 


1 


29 


43 


3 


23 


45 


9 


32 


42 . 


3 


285 


370 


44 
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We are focusing more on school improvement processes as opposed to monitoring or auditing 
of regulations. We are focusing more on data-driven decision making and helping districts to 
implement this, rather than focusing on compliance. We don’t even call it monitoring 
anymore; we call it a quality review. We are also focusing more on integration of programs to 
meet student needs. (Goals 2000) 

[Through monitoring visits] we are trying to show districts how they can coordinate and 
consolidate their professional development funding. With federal programs, they often use 
money the same way over years. We are trying to get them to look at new ways of using their 
funding to maximize it. (Title I and Title VI) 



Frequency of Monitoring 

As was true in 1996-97, with downsizing of state administrative staffs, monitoring visits 
continued to be infrequent. Indeed, 1998 survey data show that more and more programs are visiting 
fewer and fewer subgrantees in a 12-month period. That is, in addition to the majority of 
administrators of Safe and Drug-Free Schools and Communities, Eisenhower Professional 
Development, Title VI, and Neglected or Delinquent saying that fewer than one-fourth of their 
subgrantees received monitoring visits in the past 12 months (a finding that did not change from 1996- 
97), half or more of the Title I, Goals 2000, and TLCF program managers reported visiting 
subgrantees just as seldom (Table 18). In addition, about one-quarter of the respondents for 
Eisenhower Professional Development, Goals 2000, and TLCF reported that no subgrantees had 
received monitoring visits in the past 12 months, although about half these respondents specified that 
visits were planned for the future. It did appear, however, that some program administrators who said 
they did not visit any subgrantees in 1996-97 (Title VI and Neglected or Delinquent) were beginning to 
conduct monitoring visits— albeit infrequently — by 1998. 

To compensate for the staffing shortages and consequent weakening of state monitoring 
systems, some programs— recognizing the need both to know what their subgrantees are doing and to 
communicate a clear message about program purposes and goals— developed alternatives to the 
monitoring visit. For example, many state administrators had developed district self-assessment guides 
that help districts identify their own problems and needs: 

We’ve changed the monitoring instrument so that all districts can use it to do an internal review 
rather than wait for us to come out. It includes principles of effectiveness. (Safe and Drug- 
Free Schools and Communities) 
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Table 18 



Project Monitoring: Proportion of 
Program Subgrantees Receiving Monitoring Visits 
in the Past 12 Months, by Program 
(N=433) 




Number of State Administrators Who Report the Following Proportions of 
Subgrantees Receiving Monitoring Visits : 

None, But 
Plan to 
Conduct 



Federally Funded 
Education Programs 


Half or More 


Between 
Va and Vi 


Fewer 
than Va 


Visits in 
the Future 


None 


Other 


Title I, Part A (N=49) 


5 


18 


21 


2 


2 


1 


Safe and Drug-Free Schools 


and Communities (N=48) 


3 


14 


23 


2 


5 


1 


Goals 2000 (N=45) 


10 


11 


11 


5 


7 


1 


Technology Literacy 


Challenge Fund (N=49) 


12 


10 


10 


7 


7 


3 


Title VI (N=48) 


4 


17 


18 


3 


3 


3 


Eisenhower Professional 


Development (N = 50) 


3 


17 


17 


5 


6 


2 


Education of Migratory 


Children (N=47) 


24 


7 


6 


2 


4 


4 


Even Start Family 


Literacy (N = 50) 


30 


8 


7 


3 


1 


1 


Programs for Neglected or 
Delinquent Children and 
Youth (N=47) 


11 


11 


16 


2 


6 


1 


TOTAL 


102 


113 


129 


31 


41 


17 
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A self-assessment instrument that incorporates new state priorities and federal requirements has 
been prepared for school district use in determining quality and compliance across programs. 
(Title I) 

The need to know what was going on at the local level, and the state’s not being able to visit 
them as often, prompted our design of a new district self-assessment instrument. (Title I) 

Nevertheless, some administrators’ comments about the positive effects of monitoring visits on 
state and local program administration and operations illustrate why efforts to devise substitutes for 
monitoring visits appear likely to fall short: 



The monitoring process really triggers our identification of technical assistance needs and 
triggers changes to our RFP and continuation process. For example, as a result of monitoring, 
we have information about where greater clarity is required in what the expectations are, what 
the requirements are. We make some changes to our entire subgrant process. We regard the 
monitoring process as evolving and dynamic and we’re always trying to improve our effort 
based on the outcomes we see there. (Goals 2000) 

® As a result of monitoring, program directors in one state listed several areas that have 
improved, including stronger leadership, better systems of data gathering, and a 
stronger emphasis on student outcomes. The Title I director feels the most important 
result of monitoring visits has been a stronger emphasis on planning. He said: “One 
of the things we’re doing now is creating annual ‘hot topics’ with more in-depth 
questions on issues emerging as local or state priorities. This year, we are asking more 
in-depth questions about high-quality staff and services to students with limited English 
proficiency, and updating schoolwide plans.” 



Targeting of Monitoring 

As was true in 1996-97, rather than focusing their relatively infrequent monitoring activities on 
districts that might be out of compliance or where student performance was low, state program 
managers most frequently reported using a routine cycle for monitoring visits (61 percent). However, 
1998 survey data also showed that managers’ targeting methods may be starting to change: more 
administrators said they are using information about compliance problems (45 percent versus 36 
percent in the earlier survey) or student performance (21 percent versus 13 percent) to decide which 
subgrantees to visit, with no particular variation by program. However, the fact remains that few 
program managers use student performance information to target monitoring visits. This suggests, 
again, that federal efforts to promote a data-driven approach to program management have yet to pay 
off. While the percentage of Title I managers using student performance data was nearly twice as high 
as that for other programs, it still represented only 20 of 49 administrators (Table 19). 
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Table 19 



Project Monitoring: Reasons Why State Administrators 
Visit Subgrantees, by Program 
(N=433) 



On what basis is it decided which subgrantees to visit? 



Number of State Administrators Who Report That: 



Federally Funded 
Education Programs 


There is a 
Routine 
Cycle for 
Monitoring 
Visits 


Monitoring Visits 
are Triggered by 
Information That 
Grantees Are 
Having Trouble 
Meeting Program 
Reauirements 


Monitoring 
Visits Are 
Triggered by 
Information 
about Student 
Performance 


Subgrantees 
Are Visited 
at Their 
Request 


Other 


Title I, Part A (N=49) 


36 


23 


20 


16 


6 


Safe and Drug-Free Schools 


and Communities (N=48) 


24 


25 


10 


27 


11 


Goals 2000 (N=46) 


24 


21 


8 


24 


12 


Technology Literacy 


Challenge Fund (N=49) 


23 


19 


6 


28 


11 


Title VI (N=48) 


33 


25 


9 


21 


8 


Eisenhower Professional 


Development (N=50) 


23 


22 


11 


22 


5 


Education of Migratory 


Children (N=46) 


37 


14 


8 


17 


6 


Even Start Family 


Literacy (N=50) 


38 


25 


8 


22 


9 


Programs for Neglected or 
Delinquent Children and 
Youth (N=47) 


28 


21 


10 


21 


6 


TOTAL 


266 


195 


90 


198 


74 
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Integrated Monitoring Visits 



Integrated monitoring visits— monitoring visits in which a team of state program administrators 
collectively addresses the needs of several federal and state programs— continue to be the trend among 
federally funded programs; about 38 states (up from 27 in 1996-97) conducted some form of integrated 
monitoring visits. Among federal programs, integrated monitoring visits continued to be especially 
common for Title I, Title VI, Eisenhower Professional Development, and Safe and Drug-Free Schools 
and Communities programs (Table 20). A majority of administrators of Neglected or Delinquent 
programs also reported conducting integrated monitoring visits in 1998. For the most part, state 
administrators still cast a positive light on integrated monitoring visits, contending that such visits: (1) 
give a more complete picture of what subgrantees are doing by having a variety of programs 
represented (90 percent), (2) provide more solutions to subgrantees’ difficulties, also by having a 
variety of programs represented (84 percent), and (3) have reduced the cost of monitoring for their 
respective programs (66 percent) (Table 21). 

Nevertheless, some program administrators suggested that this method of monitoring, while 
generally accepted, does have its drawbacks. That is, 38 percent or more of the 213 state 
administrators who participated in integrated monitoring visits in 1998 raised many of the same 
concerns about the method as were raised in 1996-97, suggesting that the problems with integrated 
monitoring visits persist. That is, 46 percent of state administrators said the visits are too general or 
less in-depth than program- specific visits; 41 percent said the team members lacked the expertise to 
help individual programs; and 38 percent said the teams would overwhelm small districts where one 
person may administer several programs. One important development since 1996-97, however, is that 
relatively few administrators (22 percent) believed that integrated monitoring visits hampered their 
ability to collect the kind of program-specific information they needed. Migrant Education was the 
exception to this finding, with about half of program administrators reporting that integrated monitoring 
visits hampered their ability to collect program-specific information. 

Other drawbacks include the fact that, as was true in 1996-97, integrated monitoring visits are 
still not resulting in the kinds of efficiencies that would translate into visiting more sites. Again, the 
survey data showed that the programs participating in integrated monitoring visits were generally 
visiting fewer subgrantees than were those conducting program-specific monitoring visits. However, 
integrated monitoring visits do appear to be a better guarantee that at least some sites will be visited, as 
opposed to none at all. That is, only six percent of administrators participating in integrated monitoring 
visits reported that none of their subgrantees were visited in the past 12 months, compared to 26 
percent of administrators who conduct program-specific monitoring visits. 
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Table 20 



State Administrators Reporting that Their State 
Conducts Integrated Monitoring Visits, by Program 
(N=428)‘ 



Has this state conducted any integrated monitoring visits that address [this program] and other federal or state 
programs? 



Number of State Administrators Reporting that Their State: 



Federally Funded 


Conducts 
Integrated 
Monitoring Visits 
That Include 
Federally Funded 


Conducts Integrated 
Monitoring Visits with 
Both Federally and 


Does Not Conduct 
Integrated 


Does Other 


Education Programs 


Programs Only 


State-funded Programs 


Monitoring Visits 


Things 


Title I, Part A (N=49) 


19 


17 


13 


0 


Safe and Drug-Free Schools 


and Communities (N=47) 


11 


14 


22 


0 


Goals 2000 (N=44) 


9 


13 


19 


2 


Technology Literacy 


Challenge Fund (N=48) 


1 


10 


35 


2 


Title VI (N=48) 


15 


15 


13 


5 


Eisenhower Professional 


Development (N = 49) 


14 


17 


16 


2 


Education of Migratory 


Children (N=46) 


13 


8 


24 


1 


Even Start Family 


Literacy (N = 50) 


8 


14 


26 


2 


Programs for Neglected or 
Delinquent Children and 
Youth (N = 47) 


15 


11 


20 


1 


TOTAL 


105 


119 


188 


15 



1 Responses do not always sum to total because managers who answered “Don’t Know” are included in the N. 
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Table 21 

State Administrators Reporting on the Strengths and Weaknesses 
Of the Integrated Monitoring Process, by Program 
(N=213) 
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Integrated approaches to monitoring also appear to make the process more routine than do 
program-specific approaches. That is, programs participating in integrated monitoring visits are much 
more frequently reporting selecting subgrantees for monitoring visits based on a routine cycle (74 
percent compared to 49 percent conducting program-specific visits), whereas administrators who 
conduct program-specific monitoring visits are more frequently reporting visiting subgrantees based on 
information suggesting that the grantee is having trouble meeting program requirements (83 percent 
versus 49 percent participating in integrated visits). However, despite the drawbacks of integrated 
monitoring, data show that programs participating in integrated monitoring visits are more frequently 
(albeit not in very large numbers) reporting using information on student performance to target 
monitoring activities (29 percent versus 12 percent). 

Finally, as more and more states develop integrated monitoring systems, the fact that many still 
do not include some of the smaller or discretionary grant programs (e.g., Even Start, Migrant 
Education, and TLCF) continues to be a concern. As was the case for so many other administrative 
activities, program managers argued that they could not include the discretionary grant programs in the 
integrated monitoring visits because these programs: (1) awarded grants on a competitive basis and 
were therefore more interested in monitoring that focuses on program-specific issues or (2) were not 
among the programs accepting consolidated local plans or applications. Again, by excluding these 
programs, states may fail to send a coordinated, cohesive message to federal program subgrantees 
regarding standards-based reform. 



Federal Monitoring 

Early into the reauthorization period of ESEA, the U.S. Department of Education began to 
retool its program monitoring system in an effort to better support state and local education reform 
activities. Within the Office of Elementary and Secondary Education (OESE), ED established 10 
Regional Service Teams (RSTs) charged with the responsibility of monitoring and providing technical 
assistance to state and local education agencies for programs authorized and administered out of OESE. 
RSTs have “primary responsibility for cross-functional activities for the purpose of delivery of services 
to the customers of the various programs within OESE.” The 10 RSTs provide monitoring and 
technical assistance to state and local educational agencies through a variety of methods, including 
telephone contact, correspondence, and Integrated Review Team site visits. Much like state-level 
integrated monitoring visits, Integrated Review Team site visits include reviews of all elementary and 
secondary education programs in the state at one time. The goals of an integrated review are to: 

• Assist states and local school districts in determining how federal programs support 
educational reform. 
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Identify federal institutional barriers and make recommendations as appropriate. 



• Become knowledgeable about a state’s standards and assessments and the extent to 
which they promote student achievement. 

• Monitor for compliance with essential program requirements (OESE Web Page). 

As of late 1998, all but four states had been visited by a federal Integrated Review Team (IRT). 
Survey data show that among those administrators whose states had been visited by an IRT, many 
believed the visits were “somewhat” useful (37 percent); fewer respondents considered the visits to be 
“very useful” (18 percent), or “useful” (18 percent), and 19 percent said they believed that the visits 
were “not useful.” Program by program, twice as many administrators of Eisenhower Professional 
Development, Title I, Goals 2000, TLCF, and Title VI viewed the IRT visits as “very useful” than did 
administrators of other programs; however, the highest frequency of responses among these programs 
was also that the visits were only “somewhat useful” (Table 22). 

In their responses to an open-ended question asking administrators to offer their comments and 
concerns regarding these early integrated program reviews, many administrators said they were 
unimpressed by the knowledge base of ED staff and by the level of organization and focus evident in 
the visits; others complained that they had not been given enough information prior to the visit so that 
they might know what to expect; still others complained that their programs were virtually ignored by 
the IRTs. The following comments were typical: 

The Integrated Review Team’s focus was too narrow and the team appeared inexperienced and 
unprepared. They were kind of putting things together as they went along. (Title I) 

I’d like the entire process to be spelled out earlier— the year before you’re going to be 
reviewed, not in the fall before you’re reviewed. Also, it should be more systematic— using 
rating forms, for example, rather than so subjective and casual. (Eisenhower Professional 
Development and Title VI) 

It was very obvious that program reviewers didn’t know my program. (Eisenhower 
Professional Development) 

Not all programs were included; if you’re going to do integrated monitoring visits, you need to 
include all the [federal] programs. (Title I) 

Despite the problems, however, there was some good news. Some administrators believed the 
IRT visits were quite useful in that they forced programs to come together to prepare for the visits and 
thereby facilitated program communication and coordination: 
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Table 22 



State Administrators Who Believe the U.S. Department of Education’s 
Regional Service Team Visits Are a Useful Strategy for Implementing Federally Funded 
Programs to Support Comprehensive Standards-Based Reform, by Program 

(N=286) 



How useful do you think the Department’s strategy for conducting integrated reviews has been to implementing 
[this program] to support comprehensive standards-based reform? 



Number of State Administrators Who Believe 
Regional Service Team Visits Are: 



Federally Funded 
Education Programs 


Very Useful 


Useful 


Somewhat 

Useful 


Not Useful 


Don’t Know 


Title I, Part A (N = 36) 


9 


6 


13 


8 


0 


Safe and Drug-Free 
Schools and 
Communities (N = 36) 


3 


9 


11 


8 


5 


Goals 2000 (N = 30) 


7 


10 


9 


2 


2 ■ 


Technology Literacy 


Challenge Fund (N= 10) 


4 


3 


2 


1 


0 


Title VI (N=33) 


5 


4 


18 


5 


1 


Eisenhower Professional 


Development (N = 37) 


10 


4 


17 


3 


3 


Education of Migratory 


Children (N=36) 


4 


7 


14 


7 


4 


Even Start Family 


Literacy (N = 33) 


4 


6 


11 


6 


6 


Programs for Neglected or 
Delinquent Children and 
Youth (N = 39) 


5 


3 


12 


14 


1 


TOTAL 


51 


52 


107 


54 


22 
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The integrated program review is a good idea because it requires us to come together in the 
preparation for it. It helps us see how all of the federal programs fit into the larger picture. 
(Even Start) 

It promoted dialogue within our department. We did a lot of pre-planning, so we got together 
at the state level to talk. That doesn’t happen often, so that was very useful. Since then, we 
have talked about using federal funds together to achieve common goals. (Goals 2000) 



Summary: Accountability Mechanisms in Place 

After four years of implementing the law, it appears that state program administrators have 
come to recognize that supporting standards-based reform is part of their programs’ purpose. State 
administrators are also increasingly likely to point to connections between program purposes and 
student achievement. 

However, despite this progress, use of data in SEA program management remained a 
disappointment. While administrators seemed to have access to a great deal of student performance 
and program implementation data, quite a few were not using the data. Of those who were using the 
data, many reported using program implementation data for purposes of looking at issues related to 
program implementation and operations; far fewer reported using student performance data to judge 
the success of their programs in raising student achievement. Similarly, despite progress since 
1996-97, states still had far to go in building new monitoring procedures that would communicate a 
clear message about a new, standards-based accountability framework. On a more positive note, 
however, more than half of state administrators reported developing program performance indicators, 
an increase over 1996-97 levels. 

Finally, in the area of federal monitoring, state administrators’ comments about federal 
Integrated Review Team visits suggested that this method of federal monitoring has not yet fulfilled its 
promise as a method by which to “facilitate, encourage, and assist state education reform efforts to 
improve the performance of all students” (Office of Elementary and Secondary Education web page). 
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IV. Technical Assistance: Are States Working Strategically to 
Build Local Capacity in Support of Standards-Based Reform? 



According to ED, in order to ensure that all children meet high standards, the quality of 
teaching must be raised, and districts and schools “may need information and assistance” (ED, 1996a) 
in order to raise it. Federal program administrators could try to influence efforts to raise the quality of 
teaching and learning by strategically allocating technical assistance resources to inform, support, and 
promote districts’ efforts to move toward standards-based reform. 

In this chapter, we discuss the extent to which federal program offices in SEAs were bringing 
their resources to bear on building local capacity to improve teaching in support of standards-based 
reform, by comparing state-level technical assistance efforts in 1996-97 to those in 1998. In addition, 
we assess the extent to which states view other sources of assistance, particularly the U.S. Department 
of Education and its external technical assistance providers, as helpful to their efforts to build state- and 
local-level capacity in support of standards-based reform. 



State Technical Assistance 



As was true in 1996-97, agency downsizing in many SEAs continued to adversely affect the 
technical assistance capacity in federal programs. Sixty-three percent of all administrators surveyed in 
1998 reported that their SEA had been downsized or reorganized in a way that affected the staffing of 
their program (Table 23). As a result, the majority of administrators (65 percent) reported only being 
able to meet their subgrantees’ technical assistance needs “to some extent” or “a little” (Table 24). 
Administrators said: 



Overall, an issue here has been the decrease in staff. . . we’ve worked with quite a reduction 
this year; we have had three people doing what eight people used to do. Because of that, we 
have not been able to offer a lot of conferences, workshops, and things like that. We’ve been 
taking care of the administrative load only. (Title I) 

Lack of staff affects the program primarily in that I don’t always feel like I am providing the 
best technical assistance to districts because of limited staff and time. (Goals 2000) 

We’ve been reduced from two bureaus to two people. We cannot adequately meet our own 
responsibilities for administering the program much less meet our technical assistance 
responsibilities to the LEAs. The reorganization [of the SEA] made us less ‘in tune’ with the 
subgrantees. There is mass confusion within the state office. (Safe and Drug-Free Schools and 
Communities) 




71 



100 



Table 23 



State Education Agency Downsizing 
or Reorganization, by Program 
(N=430) 



Since the reauthorization of ESEA in 1994, has your SEA been downsized or reorganized in a way that has 
affected the staffing of this program? 


Federally Funded Education Programs 


Number of State Administrators Reporting 
That Their State Education Agency Has Been 
Downsized or Reorganized in a Way That 
Has Affected the Staffing of Their Programs 


Title I, Part A (N= 48) 


37 


Safe and Drug-Free Schools and Communities (N=46) 


33 


Goals 2000 (N=46) 


22 


Technology Literacy Challenge Fund (N=49) 


27 


Title VI (N=47) 


33 


Eisenhower Professional Development (N = 50) 


37 


Education of Migratory Children (N=47) 


28 


Even Start Family Literacy (N = 50) 


29 


Programs for Neglected or Delinquent 
Children and Youth (N=47) 


27 
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Table 24 



Extent to Which Programs Were Able to Meet Their Subgrantees ’ 
Technical Assistance Needs in the Past 12 Months, by Program 

(N=396) 



In your estimation, to what extent has your program been able to meet your subgrantees ’ technical assistance needs in the past 
year? 



Number of State Administrators Reporting That, 
in the Past 12 Months, They Were Able to Meet 
Their Subgrantees’ Technical Assistance Needs : 



Federally Funded 
Education Programs 


To a Great Extent 


To Some Extent 


A Little 


Not at All 


Title I, Part A (N=49) 


14 


31 


4 


0 


Safe and Drug-Free Schools 
and Communities (N=48) 


14 


27 


7 


0 


Goals 2000 (N=42) 


14 


25 


3 


0 


Technology Literacy Challenge 
Fund (N=45) 


13 


27 


5 


0 


Title VI (N=46) 


19 


25 


2 


0 


Eisenhower Professional 
Development (N = 50) 


19 


27 


4 


0 


Education of Migratory 
Children (N=40) 


15 


23 


2 


0 


Even Start Family 
Literacy (N=44) 


20 


23 


1 


0 


Programs for Neglected or 
Delinquent Children and 
Youth (N = 32) 


9 


17 


6 


0 


TOTAL 


137 


225 


34 


0 





r\ 

£ 



73 



With limited resources, state program administrators continued to make difficult choices about 
where to provide assistance. When asked about technical assistance needs that have gone— to a greater 
or lesser extent— unmet, state administrators most often said they are unable to get to every district that 
needs or requests help or that they are unable to provide sustained assistance or follow-up: 

The lack of staff inhibits our ability to meet all needs. The technical assistance is available if 
they call. Because of the size of the state, I am not sure how consumer- friendly the 
bureaucracy is. (Title VI) 

One difficulty is spreading. ; . effective ideas. There is a lack of staffing to bring people 
together and maintain communication. There is a great gap between successful and 
unsuccessful districts. Some continue to flail, while others are making changes and 
improvements that are working. Because communication of effective programs and practices 
requires staff for managing and organizing [the information], this is partly a funding issue. 

More state funds would allow us to develop and maintain a better system of communication. 
(Goals 2000) 

There is a huge need regarding the implementation of the curriculum frameworks; the need 
outweighs the availability of funds and staff. (Goals 2000) 

Technical assistance needs exceed the program’s capacity to meet them. I know it by the 
number of phone calls, what districts are asking for, and the number of schools we identified as 
in need of improvement, relative to the number of staff. Even the integrated review team told 
us we don’t have enough staff to get the job done. We are being asked to provide technical 
assistance and in every program they want to see a comprehensive, coordinated effort, but 
some people just don’t know how to do it. That is what takes more time; we’re no longer just 
doing a checklist for our program review, but you go in and visit and there’s also the follow- 
up. (Title I) 

In general, it seems that increasing numbers of state administrators (81 percent, up from 72 
percent in 1996-97) were relying on districts to know when they need help and how to ask for it, rather 
than actively assessing the local need for assistance in implementing their programs; The less 
frequently cited priorities for allocating a program’s technical assistance resources tended to be those 
types of priorities that would require states to collect their own needs assessment data. That is, far 
fewer administrators report selecting districts that have: (1) low achievement, (2) less experienced 
program managers, or (3) program compliance problems. Nevertheless, more administrators gave 
priority to these areas in 1998 than did in 1996-97. That is, 55 percent of respondents in 1998 gave 
priority to districts with low achievement, compared to only 37 percent of respondents in 1996-97. In 
addition, 55 percent of administrators in 1998 gave priority to districts with program compliance 
problems, compared to 40 percent in 1996-97 (Table 25). 
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State program administrators also faced choices about the content to emphasize in their 
technical assistance. A positive development in 1998 was that program administrators showed greater 
coordination in the content of the technical assistance they provided than in 1996-97 and much of that 
content focused on a standards-based reform agenda. Two-thirds or more of the respondents for Title 
I, Goals 2000, Title VI, and Eisenhower Professional Development said they funded or directly 
provided technical assistance to subgrantees on standards, assessment, use of data-driven decision 
making, use of technology, program coordination, and specific academic subjects. With the exception 
of program coordination, however, administrators of Safe and Drug-Free Schools and Communities, 
TLCF, Migrant Education, Even Start, and Neglected or Delinquent programs less frequently reported 
offering help to subgrantees in areas related to standards-based reform. In fact, these programs 
appeared to maintain a technical assistance agenda that remained largely responsive to program-specific 
issues. For example, the most frequently cited technical assistance focus among administrators of Safe 
and Drug-Free Schools and Communities was school safety (94 percent); for TLCF, the focus was 
educational technology (86 percent); for Even Start and Migrant Education, it was working with 
families (82 and 74 percent, respectively) (Table 26). 

The factors that most influenced state offices’ selection of technical assistance topics included 
(1) program purposes and goals (59 percent) and (2) state goals and priorities (56 percent) (Table 27). 
Goals 2000, TLCF, Title VI, and Eisenhower Professional Development administrators were well 
above the average in relying on state goals and priorities to influence their selection of technical 
assistance topics. This is probably a function of the fact that these programs tend not to focus on the 
needs of a particular population and are thereby the most flexible in the way they set their priorities; 
moreover, these are also the program administrators who— in their responses to many open-ended 
questions— tended to be most attuned to the goals and priorities of their respective states. The 
following comments were typical among this group of administrators: 

We budgeted for the state content specialists for math and science using Eisenhower 
Professional Development and Title VI funds. We also have a strategic plan that addresses 
staff development to implement standards. We also have a project team at the state level to 
help with staff development, including implementing standards and assessments. (Eisenhower 
Professional Development and Title VI) 

I think that our modifying the program for our second application to focus more on state 
priority areas is working well. People are really asking, ‘Is this project really going to have an 
impact on student performance in the long-run?’ The provisions that allow us the flexibility to 
tailor the program according to our state needs have been very helpful. (TLCF) 

We have our comprehensive state plan; it’s an umbrella for our reform goals. Our programs 
are supporting those goals. We’re connecting instructional techniques with the new standards. 

(Eisenhower Professional Development and Title VI) 



76 



ERIC 




Table 26 

Technical Assistance Topics States Fund or 
Directly Provide to Subgrantees, by Program 
(N=432) 
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Table 27 

Factors That Most Influenced State Administrators’ Selection of Technical 
Assistance Topics That Their Program Funded or Directly Provided to Subgrantees, by Program 

(N=386) 
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Few state administrators reported being influenced by program evaluation data in selecting 
technical assistance topics that their program would fund or directly provide to subgrantees (11 
percent). Even Start was above average in this category (33 percent) most likely because subgrantees 
are required to conduct annual program evaluations, thereby making evaluation data readily available 
to state Even Start coordinators. Similarly, few state administrators made use of state student 
assessment data (15 percent); the highest percentages were among administrators of Eisenhower 
Professional Development (23 percent) and Goals 2000 (22 percent). Finally, few administrators 
reported that their selection of technical assistance topics was guided by goals arising from consolidated 
planning (19 percent). 



Pooling Professional Development Funds 

Pooling funds across programs can potentially improve efficiency in the provision of 
professional development. In 1998, the vast majority of state administrators of every program except 
Neglected or Delinquent encouraged districts to pool funds for professional development across federal 
programs (Table 28). Overall, 72 percent of all administrators did so, representing no change in the 
overall percentage since 1996-97. Although the percentage varied somewhat by program, the degree 
of variation was decidedly less than in 1996-97. Again, as was true in 1996-97, the 1998 survey data 
suggest that there was some correlation between state-level efforts to coordinate and collaborate and 
whether program managers encouraged the pooling of professional development funds. For example, 
those administrators who reported coordinating administrative and operational activities or 
consolidating their administrative funding were more often reporting encouraging pooling of 
professional development funds. However, where separate subgrant applications were required, 
pooling of funds was less likely to be encouraged. 

Again, as was true in 1996-97, state administrators within a state were not consistent in the 
messages they sent districts about pooling funds for professional development. In many states, 
administrators’ efforts to encourage their subgrantees to pool professional development funds with 
particular programs were not reciprocated by those programs’ administrators— i.e., the other programs’ 
administrators said they did not encourage their subgrantees to pool funds. In other states, 
administrators reported encouraging their subgrantees to pool funds with one program while that 
program’s administrator encouraged his or her subgrantees to pool their resources with another 
program. Again, Title I was the program most often cited as a program with which other programs 
should pool their funds. 
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Table 28 



Number of State Administrators Who Encouraged Subgrantees to Pool Their 
Program Funds for Professional Development with Those of Any Other 
Federal or State Program, by Program 
(N=434) 



In the past year, did any of your written or personal communications encourage subgrantees to pool [this 
program’sl funds for professional development with those of any other federal or state program(s)? 


Federallv Funded Education Programs 


Number of State Administrators Who Encouraged 
Subgrantees to Pool Their Program Funds for 
Professional Development with Those of 
Any Other Federal or State Proeram: 


Title I, Part A (N=49) 


41 


Safe and Drug-Free Schools and 
Communities (N=48) 


29 


Goals 2000 (N = 46) 


37 


Technology Literacy Challenge 
Fund (N = 49) 


40 


Title VI (N=48) 


35 


Eisenhower Professional 
Development (N = 50) 


42 


Education of Migratory Children (N=47) 


29 1 


Even Start Family Literacy (N = 50) 


39 


Programs for Neglected or Delinquent 
Children and Youth (N=47) 


22 


TOTAL 


314 
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Local Perspectives on State Technical Assistance 



If state administrators rely on districts to know when they need help and how to ask for it, they 
can only speculate as to whether they are meeting the full range of their subgrantees technical 
assistance needs. In site visits to 16 districts in eight states, our interviews with local program 
administrators generally confirmed state managers’ worst fears: states are not meeting their 
subgrantees’ technical assistance needs. Most local administrators refer to the tremendous turnover 
their SEAs have experienced in the past several years and mourn the loss of institutional knowledge; 
some feel they are having to spend time training their state administrators to do their jobs. 

There’s heavy turnover at the SEA; we’re constantly dealing with different people. You can’t 
have major improvement without the hands to get it done; you need people to get results. They 
can’t respond as fully or as quickly because they don’t have the staff. 

Because the SEA has seen so much turnover, there’s only so much institutional knowledge 
from which districts [can] benefit. The SEA doesn’t pay enough, turnover is great. There 
have been four different state Eisenhower Professional Development coordinators since I’ve 
been here, so I know more [than they do] and it takes them a while to catch up. 

• Some local administrators in this state said that they did not see state program 
administrators as a resource to them. In fact, they perceived them to be more 
restrictive than the federal government and resistant to collaboration. Furthermore, the 
application for consolidation was seen by local administrators as illogical and a 
disincentive. 

Local-level perspectives on state administration were not all bad: some district officials did 
believe their state was giving them the assistance and support they needed: 

I talk with the state program administrator every week and he has been very helpful. He has 
provided grant writing workshops to help leverage funds, helped me create a plan for 
disseminating products, encourages collaboration, and has facilitated networking around the 
state. 



Using Federal Funds to Support State Education Reform Goals and Build Local Capacity 

The vast majority of administrators reported using their program funds to support state 
education reform goals, and administrators’ responses to a series of open-ended questions indicated that 
the kind of support some programs are funding is beginning to appear targeted toward the purpose of 
improving local capacity in support of standards-based reforms. However, the data clearly show that 
some programs provide more support than others, and are more likely to provide a kind of support that 
is more directly related to improving local-level capacity to teach to the standards and improve student 
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achievement. That is, nearly 100 percent of the administrators for Goals 2000, TLCF, Title VI, and 
Eisenhower Professional Development reported that they use their program funds to support state 
reform goals— higher than the percentage of administrators reporting for Safe and Drug-Free Schools 
and Communities (74 percent), Migrant Education (79 percent), Even Start (70 percent), and Neglected 
or Delinquent (66 percent) (Table 29). When asked about the nature of the support they provide, 
administrators of Goals 2000, TLCF, Title VI, and Eisenhower Professional Development tended to 
speak in frank terms about standards-based assistance provided to districts, schools, and teachers: 

Eisenhower funds have been used to train teachers in the use of data from the SAT test and test 
interpretation in order to improve student achievement. We’ve done this for two years at the 
district level. I think this training has helped teachers to do this and identify where students 
need more help. (Eisenhower Professional Development) 

Goals 2000 money has helped develop an assessment for new teachers with the expectation that 
teachers will be able to teach reading that is coordinated with state standards. This coming 
year, there will be an emphasis on tying reading preparation programs and support for new 
teachers— especially in reading— to standards. (Goals 2000) 

We supported staff development personnel at various educational service units, directly 
working on the reading and writing standards, both in the development of options that were 
presented to the board and worked [by] teams of teachers in schools to test various instructional 
strategies and in the review of literature that will help teachers understand what best practices 
are. This summer, in conjunction with a project funded by NSF, we took the lead in 
organizing training and discussion related to the question: How do you take standards, adjust 
curriculum, then choose assessment options? (Title VI) 

Administrators of some of the other programs said that while they believe their programs’ 
funds support state reform goals, their first priority is to fulfill the needs of the populations they serve. 
For these programs, particularly Migrant Education, Even Start, and Neglected or Delinquent, which 
do not receive much administrative funding to begin with, administrators said they are usually not able 
to focus on issues that fall outside the specific purposes and priorities of the programs they administer. 
As one administrator explained, “My program can’t support the development of standards or 
assessments because Migrant’s budget is too small.” Many state administrators of Safe and Drug-Free 
Schools and Communities said that their time and money have been completely absorbed by the 
requirement to implement the Principles of Effectiveness; the following comment was typical: “It’s 
hard to find effective programs when you’re looking at prevention. . . there are a limited number of 
evaluated programs available. The cost and time available to implement or evaluate is limited, and 
that’s problematic.” 
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Table 29 



Using Federal Education Program Funds 
to Support State Reform Goals, by Program 
(N=430) 



In your estimation, are your program funds used to support state reform goals? 


Federally Funded Education Programs 


Number of State Administrators 
Reporting That They Have Used Their 
Program Funds to 
SuDDort State Reform Goals 


Title I, Part A (N = 48) 


44 


Safe and Drug-Free Schools and 
Communities (N=46) 


34 


Goals 2000 (N = 46) 


45 


Technology Literacy Challenge 
Fund (N=49) 


48 


Title VI (N = 47) 


46 


Eisenhower Professional Development (N = 50) 


49 


Education of Migratory Children (N = 47) 


37 


Even Start Family Literacy (N = 50) 


35 


Programs for Neglected or Delinquent 
Children and Youth (N=47) 


31 
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Federal Technical Assistance 



To help states use the provisions of the new laws as effectively as possible, the U.S. 
Department of Education offers program administrators a range of sources of information and technical 
assistance. Survey data show that state program managers are rather enthusiastic about the direct 
support they receive from the U.S. Department of Education but are less satisfied with ED’s external 
technical assistance support system such as the Regional Educational Laboratories or the 
Comprehensive Centers. Indeed, after ED, state administrators are more frequently reporting 
considering other states and professional associations as being “very helpful” or “helpful” in informing 
their understanding of federal legislative provisions affecting their programs than the Regional 
Educational Laboratories or Comprehensive Centers. Specifically, survey data show that the vast 
majority of state administrators consider written information from ED (90 percent) and other contacts 
with ED through conferences, workshops, online services, and the like (83 percent) to have been “very 
helpful” or “helpful” in informing their understanding of federal legislative provisions affecting their 
programs; the next most frequent responses were other states (55 percent) and professional associations 
(38 percent). Only about one quarter of state administrators reported that they considered the 
Comprehensive Centers (25 percent) or Regional Educational Laboratories (24 percent) to be “very 
helpful or helpful (Table 30). Program by program, Title I administrators were most frequently 
reporting that the Regional Educational Laboratories were helpful, but they still numbered fewer than 
half. Title I was also the outlier with respect to the Comprehensive Centers, with 23 of the 49 state 
directors believing the centers were “very helpful” or “helpful”; numbers well above the average. At 
the other end of the spectrum, TLCF administrators were those least likely to view the centers as 
helpful, with only two of the 49 TLCF managers selecting this response. 

Regarding the timeliness with which state administrators received information from the U.S. 
Department of Education about the federal legislative provisions affecting their programs, the majority 
of state administrators— with no significant variation by program— said the information they received 
was “somewhat” or “very” timely (73 percent). Among the 25 percent of respondents who reported 
that the information they received from ED was “somewhat” or “very” slow, Safe and Drug-Free 
Schools and Communities and the Technology Literacy Challenge Fund were slightly above the 
average, with about one-third of respondents selecting these response options (Table 31). 

Finally, when asked which sources of guidance are generally the most influential when state 
administrators are making decisions about program administration, the vast majority cited the U.S. 
Department of Education (92 percent). The next most influential source of guidance— with no 
significant variation by program— was “key policymakers within the SEA” (66 percent). Interestingly, 
more administrators cited “local districts or other subgrantees” (41 percent) as a source of guidance 
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State Administrators Who Find Various Organizations and 
Agencies to be Helpful in Informing Their Understanding of 
Federal Legislative Provisions Affecting Their Program, by Program 

(N=431) 
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Table 31 



Timeliness with Which State Administrators Received 
Information from the U.S. Department of Education Regarding the 
Federal Legislative Provisions Affecting Their Program, by Program 

(N=430) 



In the past year, how would you rate the timeliness with which you have received information from the U.S. Department of 
Education regarding the federal legislative provisions affecting your program? 



Number of State Administrators Reporting That the Timeliness with Which They 
Received Information from the U.S. Department of Education Regarding the 
Legislative Provisions Affecting Their Program Was: 



Federally Funded 
Education Programs 


Very Slow 


Somewhat Slow 


Somewhat Timely 


Very Timely 


Don’t Know 


Title I, Part A (N=48) 


2 


9 


23 


14 


0 


Safe and Drug-Free Schools 
and Communities (N = 47) 


8 


9 


24 


5 


1 


Goals 2000 (N=46) 


3 


3 


20 


19 


1 


Technology Literacy Challenge 
Fund (N=48) 


5 


11 


15 


17 


0 


Title VI (N=48) 


8 


5 


23 


11 


1 


Eisenhower Professional 
Development (N=50) 


3 


9 


26 


12 


0 


Education of Migratory 
Children (N=47) 


4 


4 


22 


16 


1 


Even Start Family 
Literacy (N=50) 


3 


8 


24 


13 


2 


Programs for Neglected or 
Delinquent Children and 
Youth (N = 46) 


4 


10 


23 


9 


0 


TOTAL 


40 


68 


200 


116 


6 




86 
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than “other state administrators of federal programs within the state” (36 percent) or “state 
administrators of federal programs in other states” (36 percent), suggesting that state administrators are 
paying attention to the particular needs of local school districts when they make decisions about 
program administration, rather than focusing on what other state administrators or other states do 
(Table 32). 

Summary: Positive Steps, Insufficient Capacity 

As was true in 1996-97, agency downsizing in many SEAs continued to adversely affect the 
technical assistance capacity in federal programs, and state administrators continued to rely on districts 
to know when they need help and how to ask for it. 

A positive development in 1998 was that program administrators showed greater coordination 
in the content of the technical assistance they provided than in 1996-97, and much of that content 
focused on a standards-based reform agenda. Indeed, the vast majority of administrators reported using 
their program funds to support state education reform goals, and administrators’ responses to a series of 
open-ended questions indicated that the kind of support some programs are funding is focused on 
improving local capacity in support of standards-based reforms. 

However, despite these developments, with the exception of the Even Start program, few state 
administrators reported being influenced by program evaluation data in selecting technical assistance 
topics that their program would fund or directly provide to subgrantees. Most disturbingly, our 
interviews with local program administrators generally confirmed state managers’ fears: at least in this 
small sample of districts from around the country, states are not meeting their subgrantees’ technical 
assistance needs. 
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Table 32 



Sources of Guidance That State Administrators Report Are the Most 
Influential Regarding Decisions About Program Administration, by Program 

(N=432) 



When you are making decisions about how to administer [this program], which of the following sources of guidance are generally the most 
influential? 



Number of State Administrators Reporting That 
the Following Sources of Guidance Are the Most Influential: 



Federally 

Funded 

Education 

Programs 


U.S. 

Department of 
Education 


Key 

Policymakers 
in the SEA 


Other State 
Administrators 
of Federal 
Programs in 
This State 


State 

Administrators of 
Federal Programs 
in Other States 


Multi-State 
Providers of 
Technical 
Assistance 


Local Districts 
or Other 
Subgrantees 


Other 


Title I, Part A 
(N=49) 


48 


33 


15 


19 


10 


18 


1 


Safe and 
Drug-Free 
Schools and 
Communities 
(N=47) 


46 


30 


19 


12 


7 


24 


3 


Goals 2000 
(N=46) 


36 


34 


18 


2 


7 


26 


7 


Technology 
Literacy 
Challenge 
Fund (N=48) 


43 


35 


13 


12 


4 


22 


8 


Title VI 
(N=48) 


41 


29 


26 


25 


4 


13 


2 


Eisenhower 
Professional 
Development 
(N = 50) 


49 


37 


27 


15 


3 


15 


2 


Education of 
Migratory 
Children 
(N=47) 


45 


29 


11 


27 


7 


20 


2 


Even Start 
Family 
Literacy 
(N=50) 


45 


27 


15 


27 


6 


20 


5 


Programs for 
Neglected or 
Delinquent 
Children and 
Youth (N=47) 


46 


32 


13 


16 


6 


17 


2 




V. Performance Indicators: Are States Collecting and Using 
Indicators Data to Inform Program Performance? 



Findings from the baseline study showed that program performance indicators were not a 
particularly common means by which program administrators assessed and improved program success. 
Since that study was conducted, the U.S. Department of Education has developed— as mandated under 
the Government Performance and Results Act— a set of program performance indicators intended to 
inform Congress, the Department, and the nation about the effectiveness of federal elementary and 
secondary programs. In 1998, state administrators were again asked whether they had developed their 
own set of program performance indicators and how those indicators were used. In addition, they were 
asked whether they: (1) were aware of the program performance indicators developed by the U.S. 
Department of Education and (2) collected and used the federal performance indicator data. 



State-Developed Program Performance Indicators 



Program performance indicators appear to be on the rise among federally funded programs. In 
1998, 55 percent of state administrators reported that their program either had developed (26 percent) 
or was in the process of developing (29 percent) performance indicators; in 1996-97, the overall 
percentage was just 41 percent. Performance indicators were found in significant majorities in state 
offices administering Safe and Drug-Free Schools and Communities (85 percent) and Eisenhower 
Professional Development (84 percent). About half the administrators of TLCF reported having 
developed or being in the process of developing performance indicators, and slightly less than half of 
the offices administering Title I, Even Start, Title VI, and Neglected or Delinquent programs reported 
doing so. Migrant Education administrators posted the lowest numbers, with only 34 percent of state 
administrators reporting having or being in the process of developing performance indicators (Table 
33). 



Among the 26 percent of state administrators who reported having already developed— as 
opposed to being in the process of developing — performance indicators, most reported that they were 
using the indicators to inform their work and to plan and coordinate across programs. Specifically, state 
administrators reported using the indicators to: (1) help program staff focus on program goals and 
objectives (82 percent), (2) provide program outcome data for administrative planning (74 percent), (3) 
monitor the progress of local programs or subgrantees (72 percent), and (4) provide state and local 
administrators with a common language for defining program results (67 percent) (Table 34). 
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Table 33 



State Administrators Reporting That Their Program 
Has Developed Performance Indicators, by Program 
(N=385) 



Has this program developed performance indicators? 






Number of State 


Number of State 






Administrators 


Administrators 






Reporting That 


Reporting That 






Their Program 


Their Program 






HAS DEVELOPED 


IS DEVELOPING 






Performance 


Performance 


Federally Funded Education Programs 


Indicators 


Indicators 


Title I, Part A (N=47) 


8 


15 


Safe and Drug-Free Schools and Communities (N=47) 


28 


12 


Technology Literacy 


' Challenge Fund (N=50) 


5 


21 


Title VI (N=48) 




6 


14 


Eisenhower Professi' 


onal Development (N=50) 


30 


12 


Education of Migratory Children (N=47) 


6 


10 


Even Start Family Literacy (N=49) 


10 


14 


Programs for Neglected or Delinquent 






Children and Youth (N = 47) 


7 


12 


TOTAL 




100 


110 




123 
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State Administrators Reporting on How 
They Use Their Program Performance Indicators, by Program 

(N=99) 
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Federal Program Performance Indicators 

Federal program administrators responded to questions about their awareness of the existence 
of federal program performance indicators that the U.S. Department of Education developed in 
response to the Government Performance and Results Act. In particular, administrators were asked 
about the extent to which they collect data from local school districts on selected indicators, and (in 
cases where data are collected) how this information is used. While some programmatic differences 
were identified, no state-level patterns emerged. That is, consistent responses were not found across all 
programs in any of the states. The following discussion focuses first on trends found across all 
respondents and then moves to program-specific analyses of data in each of the following areas: 
awareness of performance indicators, data collection efforts, uses of program performance indicator 
data, and, for programs that did not collect data, reasons why these data were not collected. 

Overall, about 62 percent of all state administrators reported being aware of the fact that the 
U.S. Department of Education had developed a set of performance indicators for their respective 
programs. The numbers of state administrators expressing awareness, however, did vary significantly 
by program. That is, administrators of Eisenhower Professional Development (98 percent), Safe and 
Drug-Free Schools and Communities (85 percent), and the Technology Literacy Challenge Fund (79 
percent) were well above average in terms of knowing about the existence of the federal program 
performance indicators; conversely, administrators of Migrant Education (38 percent), Goals 2000 (35 
percent), and Neglected or Delinquent programs (34 percent) were well below the average (Table 35). 
Some of this variation can be explained by the fact that at the time the survey was administered, ED 
had only recently transmitted the indicators to the states. It is likely that information about the 
indicators had not yet filtered down to the relevant program managers. 

Of the 267 program administrators who were aware of the federal program performance 
indicators, 217 or 81 percent also collected some or all data on them. As illustrated in Table 36, 
administrators of Safe and Drug Free Schools and Communities and the Technology Literacy Challenge 
Fund reported both being aware of and collecting these data more than any other program (100 percent 
and 92 percent of administrators collected data, respectively). 

Of the 217 program administrators who collected some or all data related to the federal 
program performance indicators, 209 responded to questions about the corrective actions they take 
when local school districts fail to submit these data to the state. The survey data show that most 
administrators appeared to value the indicators as a source of information and to be willing to ensure 
that such data are provided. That is, most of the 209 administrators (77 percent) reported initiating 
some type of corrective action against local school districts that fail to submit the data to the state. 
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Table 35 



State Administrators Reporting That They 
Are Aware of the U.S. Department of Education’s 
Program Performance Indicators, by Program 
(N=429) 



In response to the requirements of the Government Performance and Results Act, the U.S. Department of 
Education (ED) has developed a set of performance indicators for [your] program. Are you aware of the 
indicators? 


Federallv Funded Education Programs 


Number of State 
Administrators 


Title I, Part A (N= 47) 


28 


Safe and Drug-Free Schools and Communities (N=48) 


41 


Goals 2000 (N=46) 


16 


Technology Literacy Challenge Fund (N=48) 


38 


Title VI (N=48) 


31 


1 

Eisenhower Professional Development (N = 50) 


49 


Education of Migratory Children (N=47) 


18 


Even Start Family Literacy (N=48) 


30 


Programs for Neglected or Delinquent 
Children and Youth (N=47) 


16 


TOTAL 


267 
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Table 36 



State Administrators Reporting That They 
Collect Program Performance Indicator Data, by Program 

(N=267) 



To assist in updating and improving ED*s performance indicators, we are attempting to gauge the extent to which 
the following information is collected by your state. Please provide your best estimates as to the availability of 
the following information. 


Federallv Funded Education Programs 


Number of State Administrators That 
Collect Data from All Districts, or 
from a Sample of Districts on 
At Least One Indicator 


Title I, Part A (N=28) 


21 


Safe and Drug-Free Schools and Communities (N = 41) 


41 


Goals 2000 (N=16) 


14 


Technology Literacy Challenge Fund (N = 38) 


35 


Title VI (N = 31) 


24 


Eisenhower Professional Development (N=49) 


31 


Education of Migratory Children (N= 18) 


14 


Even Start Family Literacy (N = 30) 


24 


Programs for Neglected or Delinquent 
Children and Youth (N = 1 6) 


13 


TOTAL 


217 
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Of the 161 state administrators who reported initiating corrective action against school districts that fail 
to submit data related to the performance indicators, most said they do so by: (1) assisting local school 
districts in developing a plan to collect the data (63 percent), (2) withholding program funds (43 
percent), or (3) other means (16 percent) (Table 37). 

Of the 217 program administrators who collect some or all data related to the federal program 
performance indicators, most reported using these data for a variety of purposes. That is, most 
administrators reported using the data to report to federal officials (81 percent). In addition, 
administrators use these data to: (1) identify priorities for state-level technical assistance (74 percent), 
(2) identify districts and schools for which technical assistance is needed (73 percent), (3) report to state 
officials (67 percent), and (4) use in some other manner (10 percent) (Table 38). 

Finally, of the 97 program administrators who are aware of the federal program performance 
indicators but do not collect the data, many did report that it would be possible to collect these data in 
the future. That is, of the 66 program administrators who responded to questions about the feasibility 
of collecting these data in the future, 44 (67 percent) said that it would be possible to collect these data 
but with some difficulty, and 20 (30 percent) stated that they could collect these data fairly easily. 

Only 16 program administrators (24 percent) stated that it would be extremely difficult to collect this 
information, and five (8 percent) stated that it would be inappropriate to do so. 



Program-Specific Uses of Federal Performance Indicators Data 

The extent to which program administrators collected and used federal performance indicators 
data appeared to vary more by the type of indicator (each program has its own set of indicators) than 
by program. 

Title I. Twenty-eight of the 47 Title I program administrators reported that they were aware 
of the federal performance indicators. Of these, 21 administrators reported collecting data on at least 
one of the four indicators and five collected data on all four indicators: Research-based Curriculum 
and Instruction (Indicator 2.3), Qualified Teacher Aides (Indicator 2.6), Accountability: Intervention 
(Indicator 4.3a), and Accountability: Assistance (Indicator 4.3b). Furthermore, they collected data on 
the accountability indicators (Indicators 4.3a and 4.3b) approximately twice as frequently as they 
collected data on Research-based Curriculum and Instruction or on Teacher Aides (Indicators 2.3 and 
2.6, respectively). That is, out of 27 program administrators, 20 collected data on Accountability: 
Assistance; 19 administrators collected data on Accountability: Intervention; 10 administrators collected 
data on Qualified Teacher Aides; and 8 administrators collected data on Research-based Curriculum 
and Instruction. Finally, most administrators who collected these data collected them from all districts 
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Table 37 



Ways in Which States Initiate 
Corrective Action Against Local School Districts 
That Fail to Submit Data to the State on the 
Indicators for Which Data Have Been Requested or Required 

(N=209) 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local school 
districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 



Number of State Administrators Reporting That 
Their State Initiates Corrective 
Action Against School Districts by: 







Assisting Local 
School Districts 
in Developing a 


Withholding 




Number of State 
Administrators Reporting 
That Their State DOES 
NOT Initiate Corrective 


Federally Funded 


Don’t 


Plan to Collect 


Program 


Other 


Action Against Local 


Education Programs 


Know 


the Data 


Funds 


Means 


School Districts 


Title I, Part A (N=20) 


0 


7 


5 


3 


6 


Safe and Drug-Free Schools and 


Communities (N = 40) 


1 


23 


18 


7 


6 


Goals 2000 (N=14) 


0 


7 


7 


1 


3 


Technology Literacy Challenge 


Fund (N = 34) 


0 


16 


12 


4 


9 


Title VI (N= 14) 


0 


8 


5 


3 


7 


Eisenhower Professional 


Development (N = 34) 


1 


13 


10 


3 


7 


Education of Migratory 


Children (N=14) 


0 


8 


3 


1 


4 


Even Start Family Literacy 
(N=24) 


0 


14 


5 


4 


3 


Programs for Neglected or 


Delinquent Children and Youth 
(N= 1 1) 


1 


6 


4 


0 


3 


TOTAL 


3 


102 


69 


26 


48 
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Table 38 



Ways in Which States Use 
Federal Program Performance Indicator Data 
(N=217) 



[Where data are collected] If your state collects data on any of the indicators listed in Question 21, how is the 
information used? 



Number of State Administrators Reporting That They Use 
Program Performance Indicator Data from 
At Least One Indicator to: 

Identify 
Districts and 

Schools for Identify 

which Priorities for 



Federally Funded 
Education Programs 


Technical 
Assistance is 
Needed 


Report to 
State 
Officials 


Report to 
Federal 
Officials 


State-level 

Technical 

Assistance 


Other 


Title I, Part A (N=21) 


19 


13 


15 


16 


3 


Safe and Drug-Free Schools and 
Communities (N = 41) 


32 


32 


39 


31 


1 


Goals 2000 (N=14) 


11 


13 


9 


13 


1 


Technology Literacy Challenge 
Fund (N = 35) 


25 


28 


28 


22 


3 


Title VI (N=23) 


9 


9 


17 


12 


6 


Eisenhower Professional 
Development (N = 31) 


19 


15 


28 


23 


3 


Education of Migratory 
Children (N=14) 


11 


9 


11 


12 


1 


Even Start Family Literacy (N=23) 


22 


16 


17 


21 


3 


Programs for Neglected or Delinquent 
Children and Youth (N = 13) 


8 


8 


10 


9 


0 


TOTAL 


156 


143 


174 


159 


21 
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receiving program funds, rather than from a sample of districts. This held true for each of the four 
indicators-Accountability: Assistance (16 out of 20 administrators); Accountability: Intervention (14 
out of 19 administrators); Qualified Teacher Aides (9 out of 10 administrators); and Research-based 
Approaches to Improving Curriculum and Instruction (6 out of 8 administrators) (Table 39). 

Most Title I program administrators who collect performance indicator data reported taking 
some type of corrective action when LEAs failed to submit program performance indicator data. Their 
diligence in encouraging districts to collect these data may indicate that they are of some use to 
program administrators. Of the 21 program administrators who collected federal performance indicator 
data, 14 reported taking some form of corrective action against districts that fail to submit data to the 
state, including: assisting local school districts in developing a plan to collect data (7 administrators), 
withholding program funds (5 administrators), or taking some other form of corrective action (3 
administrators). Only six administrators said that they do not take any form of corrective action against 
districts that fail to submit data to them, and one administrator did not respond to this question. 

Most Title I program administrators reported using the performance indicator data that they 
collect for a variety of purposes, including: identifying schools and districts for which technical 
assistance is needed (19 administrators); identifying priorities for state-level technical assistance (16 
administrators); reporting to federal officials (15 administrators); and reporting to state officials (13 
administrators). Indicator by indicator, however, it appears that more states collect and use the data for 
the accountability indicators (Indicators 4.3a and 4.3b) than other indicators and that they use these data 
for purposes of identifying technical assistance needs and priorities (Table 40). 

Of the 22 program administrators who reported that they do not collect data on at least one 
program performance indicator or that they do not know whether they collect these data, 10 
administrators predicted how difficult it would be to collect data on at least one indicator in the future. 
Six administrators indicated that it would be possible to collect these data, but with some difficulty; five 
stated that it would be extremely difficult to collect and report these data; three said that they could 
collect and report program performance data fairly easily; and one had no knowledge. 

Even Start. Thirty of the 48 Even Start program administrators surveyed indicated that they 
were aware of the federal program performance indicators. Of these, 24 reported collecting data on at 
least one program performance indicator and 20 collected data on all three of the indicators included in 
the surveys: Parenting Skills (Indicator 1.4), Adult Literacy Achievement (Indicator 1.1), and 
Children’s Language Development and Reading Readiness (Indicator 1.3). 
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Table 39 



Extent to Which Title I Program Administrators 
Collect Data on Federal Program Performance Indicators 

(N=27) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as 
to the availability of the following information. SEA Currently Collects Data? 







SEA Currently Collects Data? 




Indicator 


Yes, from 
ALL 
districts 
receiving 
program 
funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


2.3: Research-based curriculum and instruction. 

The proportion of schools using comprehensive, 
research-based approaches to improve curriculum 
and instruction. 


6 


2 


13 


5 


1 


2.6: Qualified teacher aides. The percent of 
districts providing support for the educational 
improvement of paraprofessionals/teacher aides will 
increase. 


9 


1 


10 


5 


0 


4.3a: Accountability: intervention. States and 
districts provide assistance to schools not making 
progress (through school support teams and other 
sources). 


14 


5 


5 


1 


2 


4.3b: Accountability: assistance. States and 
districts will take appropriate action with schools 


16 


4 


4 


1 


2 



that consistently fail. 
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Table 40 



Title 1 Program Administrators’ 

Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators listed, how is 


the information used? 












Indicator 


Indicator 


Indicator 


Indicator 




2.3 


2.6 


4.3a 


4.3b 


How Data Are Used 


(N=9) 


(N = 10) 


(N = 19) 


(N=20) 


To identify districts and schools for which technical assistance is 
needed 


7 


5 


17 


17 


To report to state officials 


4 


3 


10 


10 


To report to federal officials 


5 


7 


10 


11 


To identify priorities for state-level technical assistance 


6 


4 


14 


14 


Other 


1 


1 


2 


2 
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In comparison to Title I, there was less variation in the number of Even Start administrators 
who reported collecting data on the federal program performance indicators: Even Start administrators 
collected data on each indicator nearly equally (Table 41). Specifically, 24 administrators reported 
collecting data on Parenting Skills, 22 administrators collected data on Adult Literacy Achievement, 
and 21 administrators collected data on Children’s Language Development and Reading Readiness. In l 

addition, most of the Even Start administrators reported that they collect indicators data from all 
districts receiving program funds rather than from a sample of districts. More specifically, 21 
administrators collect data from all districts for both Adult Literacy Achievement (Indicator 1.1) and 
Parenting Skills (Indicator 1.4) and 19 administrators collect data from all districts on Children’s 
Language Development and Reading Readiness (Indicator 1.3). 

Even Start program administrators also seem to value program performance indicator data and 
take corrective action when data are not submitted to the state. That is, 18 of the 24 administrators 
who collect data on at least one indicator reported taking some sort of corrective action against 
districts. Of these, 14 indicated that they work with LEAs to develop a data collection plan and five 
said that they withhold program funds. 

As with Title I, Even Start program administrators most frequently reported using performance 
indicator data to inform their technical assistance activities. That is, more Even Start administrators 
reported using the indicators data to identify schools and districts for which technical assistance is 
needed or to identify priorities for state-level technical assistance than using them to report to state or 
federal officials. By comparison, fewer Even Start administrators reported using performance indicator 
data for purposes of reporting to state and federal officials: 16 administrators for indicator 1.1, 15 
administrators for indicator 1.4, and 14 administrators for indicator 1.3 (Table 42). 

Ten Even Start program administrators said that they do not collect or do not know if they 
collect data on at least one program performance indicator, and seven predicted how difficult it would 
be to collect these data in the future. The vast majority of these administrators said that they could 
“fairly easily” or “possibly” collect performance indicator data for all three indicators. 

Migrant Education. Eighteen of the 47 Migrant Education program administrators surveyed 
said that they were aware of the federal program performance indicators, and 14 reported collecting 
data on at least one indicator. Specifically, 14 administrators reported collecting data on Program 
Coordination (Indicator 3.4) and 10 collect data on Inter- and Intrastate Coordination (Indicator 3.1). 

For each of these indicators, most administrators reported collecting data from all districts receiving 
program funds rather than from a sample of districts. That is, 12 of the 14 administrators collected 
data from all districts on Program Coordination (Indicator 3.4) and eight administrators did so on Inter- 
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Table 41 



Extent to Which Even Start Program Administrators 
Collect Data on Performance Indicators 
(N=30) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimate as to the availability of the 
following information. SEA Currently Collects Data? 



SEA Currently Collects Data? 

Yes, from 



Indicator 


ALL 

districts 

receiving 

program 

funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Adult literacy achievement. By fall 2001, 40 
percent of Even Start adults will achieve significant 
learning gains on measures of math skills and 30 
percent of adults will achieve such gains on 
measures of reading skills. 


21 


1 


8 


0 


0 


1.3: Children’s language development and 
reading readiness. By fall 2001 , 60 percent of 
Even Start children will attain significant gains on 
measures of language development and reading 
readiness. 


19 


2 


9 


0 


0 


1.4: Parenting skills. Increasing percentages of 
parents will show significant gains on measures of 
parenting skills, knowledge, and expectations for 


21 


3 


6 


0 


0 



their children. 
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Table 42 



Even Start Program Administrators’ 

Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators 
listed, how is the information used? 




Indicator 


Indicator 


Indicator 




1.1 


1.3 


1.4 


How Data Are Used 


(N=22) 


(N~21) 


(N=24) 


To identify districts and schools for which technical assistance is 
needed 


20 


19 


21 


To report to state officials 


16 


14 


15 


To report to federal officials 


16 


14 


15 


To identify priorities for state-level technical assistance 


20 


19 


20 


Other 


3 


2 


3 
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and Intrastate Coordination (Indicator 3.1) from all districts and 12 administrators did so for program 
coordination (Table 43). 

Of the 14 Migrant Education program administrators who collect performance indicator data, 
most (10 administrators) reported taking some form of corrective action against those LEAs that failed 
to submit performance indicator data to the state. As with Even Start and Title I, the most frequently 
reported corrective action that Migrant Education program administrators take is assisting local school 
districts to develop a data collection plan (8 administrators). Another corrective action that Migrant 
Education administrators reported taking is withholding program funds (3 administrators). 

Most program administrators’ used the indicators data (both Inter- and Intrastate Coordination 
and Program Coordination) to report to federal officials (9 and 10 administrators, respectively) and to 
identify priorities for state-level technical assistance (9 and 12 administrators, respectively) (Table 44). 

Seven Migrant Education program administrators said that they do not collect or do not know if 
they collect data on at least one program performance indicator, and five predicted how difficult it 
would be to collect these data in the future. Four administrators indicated that it would be possible to 
collect and report data for Indicator 3.1 but with some difficulty. 

Neglected or Delinquent Youth. Sixteen of the 47 administrators of the Neglected or 
Delinquent program who responded to the survey said they were aware of federal program 
performance indicators. Of these, 13 collected data on at least one performance indicator and seven 
collected data on all three indicators: Academic Achievement (Indicator 1.1), Institution-wide 
Programs (Indicator 2.1), and Innovative Transition Programs (Indicator 2.2). Indicator by indicator, 
administrators of the Neglected or Delinquent program reported collecting data on Academic 
Achievement and Institution-wide Programs more frequently than on Innovative Transition Programs. 
Of the 16 administrators who were aware of the performance indicators, 11 reported that they collect 
data on Academic Achievement and Institution-wide Programs, and eight program administrators said 
that they collect data on Innovative Transition Programs, as shown in Table 45. This variation in data 
collection activity may reflect the degree of difficulty administrators face in defining and collecting 
indicators data. That is, Academic Achievement and Institution-wide Programs may be more easily 
defined and measured than Innovative Transition Programs. 

As was the case with other federal programs, many state administrators of the Neglected or 
Delinquent programs took some form of corrective action against districts that failed to submit program 
performance indicator data to the state. As is the case with many other programs, more administrators 
who took corrective action used a collaborative approach to gather these data (i.e., assisting local 
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Table 43 



Extent to Which Migrant Education Program Administrators 
Collect Data on Federal Program Performance Indicators 

(N=18) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as to 
the availability of the following information. SEA Currently Collects Data? 



SEA Currently Collects Data? 



Indicator 



Yes, from 
ALL 
districts 

receiving Yes, from a 
program sample of Don’t 

funds districts No Know Other 



3.1: Inter- and instrastate coordination. SEAs 
and LEAs will demonstrate increased interstate and 
intrastate coordination to improve educational 
continuity for migrant students. Measures of 
coordination include joint products resulting from 
these formal agreements, meetings, or conferences 
to promote coordination; coordinated guidance to 
grantees; and joint planning by local staff from all 
available programs. 



8 2 6 1 1 



3.4: Program coordination. Federal, SEA, and 
LEA staff working with Title I, Part A and Part C, 
and other federally funded programs, will 
demonstrate increasing levels of substantive 
collaboration to meet the unmet needs of migrant 
children. 
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Table 44 



Migrant Program Administrators’ 

Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators listed, how is 
the information used? 


' 


Indicator 


Indicator 




3.1 


3.4 


How Data Are Used 


(N=10) 


(N = 14) 


To identify districts and schools for which technical assistance is needed 


7 


11 


To report to state officials 


7 


8 


To report to federal officials 


9 


10 


To identify priorities for state-level technical assistance 


9 


12 


Other 


0 


1 
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Table 45 



Extent to Which Administrators for the Neglected or Delinquent Program 
Collect Data on Federal Program Performance Indicators 

(N=16) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as 
to the availability of the following information. SEA Currently Collects Data? 






SEA Currentlv Collects Data? 




Indicator 


Yes, from 
ALL 
districts 
receiving 
program 
funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Academic achievement. The number of N, 
D, and at-risk children and youth who will progress 
toward a high school diploma or GED while 
institutionalized will increase 


9 


2 


4 


1 


0 


2.1: Institution-wide programs. The number of 
institutions that will operate institution-wide 
programs integrating other federal and state 
programs to improve curriculum and instruction 
across the institution will increase 


10 


1 


4 


0 


0 


2.2: Innovative transition programs. State and 
local programs will develop innovative strategies 
that help institutionalized students make a successful 
transition from an institution back to the community, 
either to further their education or to obtain 
employment. 


5 


3 


7 


1 


0 




107 



141 



school districts to write a data collection plan, used in six states) than used stronger sanctions (i.e., 
withholding program funds, done in four states). 

Although equal numbers of state administrators collected data on Academic Achievement and 
Institution-wide Programs (8 administrators), more administrators reported using data on Academic 
Achievement for such purposes as: reporting to state officials, identifying districts and schools for 
which technical assistance is needed, and identifying priorities for state-level technical assistance. Six 
administrators used these data to identify priorities for technical assistance; five administrators each 
used institution-wide data to report to state officials and to identify schools and districts that need 
technical assistance (Table 46). 

Of the nine Neglected or Delinquent program administrators who reported that they do not 
collect or do not know if they collect data on at least one program performance indicator, six responded 
to questions about how difficult it would be to collect these data in the future. Most program 
administrators (five for academic achievement, two for institution-wide programs, and five for 
innovative transition programs) indicated that it would be possible to collect these data in the future. 
However, two administrators said that it would be inappropriate to collect data on institution-wide 
programs (Indicator 2.1). 

Title II. The Eisenhower Professional Development Program had the highest proportion of 
state administrators (49 out of 50 administrators) who reported that they are aware of the Department 
of Education’s program performance indicators. Of these, 31 administrators reported collecting data 
on at least one of the performance indicators that we asked about and 19 collected data on all three 
performance indicators: Duration (Indicator 3.2), Teachers’ Skills and Classroom Instruction 
(Indicator 1.1), and High Poverty Schools (Indicator 4.1). 

Indicator by indicator, administrators of the Eisenhower Professional Development Program 
most frequently reported collecting data on the duration and impact of subgrantee activities, rather than 
on the population served. As shown in Table 47, out of 49 administrators, 28 administrators collected 
data on the duration of professional development programs; 24 collected data on teachers’ skills and 
classroom instruction; and 22 gathered data on the degree to which Title II activities are offered to 
teachers in high-poverty schools. 

Of the 31 program administrators who reported collecting performance indicator data, 23 said 
they used some type of sanction when local school districts failed to submit indicator data to them. Of 
these, 13 said they assisted local school districts to write a data collection plan, 10 withheld program 
funds, and three used some other form of action. 
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Table 46 



Neglected or Delinquent Program Administrators’ 
Use of Federal Program Performance Indicator Data 
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Table 47 



Extent to Which Administrators of the Eisenhower Professional Development Program 
Collect Data on Federal Program Performance Indicators 

(N = 49) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as to 
the availability of the following information. SEA Currently Collects Data? 






SEA Currently Collects Data? 




Indicator 


Yes, from 
ALL 
districts 
receiving 
program 
funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Teachers’ skills and classroom instruction. 

By 1998, over 50% of a sample of teachers will 
show evidence that participation in Eisenhower- 
assisted professional development has resulted in 
improvement in their knowledge and skills, and by 
1999 in an improvement in classroom instruction. 


17 


7 


23 


0 


2 


3.2: Duration. By 1998, 35% of teachers 
participation in district-level Eisenhower assisted 
activities will participate in activities that are a 
component of professional development that extends 
over the school year; by 2000, over 50% will. 


23 


5 


16 


2 


3 


4.1: High-poverty schools. The proportion of 
teachers participating in Eisenhower-assisted 
activities who teach in high-poverty schools will 
exceed the proportion of the national teacher pool 
who teach in high-poverty schools. 


19 


3 


22 


3 


2 




no 1 4 4 



Most administrators of the Eisenhower Professional Development Program used performance 
indicator data in similar ways. For example, administrators reported using performance indicator data 
to report to federal officials (23 administrators for Indicator 1.1, 26 administrators for Indicator 3.2, 
and 20 administrators for Indicator 4.1). Administrators also used specific indicators to inform a 
variety of administrative activities. For example, Teachers’ Skill and Classroom Instruction (Indicator 

1.1) was also used to: identify priorities for state-level technical assistance (19 administrators), identify 
districts and schools for which technical assistance is needed (14 administrators), and report to state 
officials (13 administrators). As shown in Table 48, similar results were found for Duration (Indicator 

3.2) and High-poverty Schools (Indicator 4. 1). 

Eisenhower Professional Development program administrators were less optimistic about their 
ability to collect data on Teachers’ Skill and Classroom Instruction (Indicator 1.1) than about collection 
of data on Duration (Indicator 3.2) or High-poverty Schools (Indicator 4. 1). That is, only 12 out of 21 
program administrators said that it would be possible to collect data on Indicator 1.1, compared to 12 
out of 15 administrators for Indicator 3.2 and 18 out of 23 administrators for Indicator 4. 1 (Table 49). 
This may reflect the difficulty of assessing the impact of professional development on instructional 
practice. In comparison, duration of professional development activities and the demographics of the 
schools served are much easier to measure. 

Technology Literacy Challenge Fund. Thirty-eight of the 48 Technology Literacy Challenge 
Fund program administrators surveyed were aware of the federal program performance indicators. Of 
these, 35 collected data on at least one indicator, and 27 collected data on both indicators: Staff 
Training (Indicator 3.3) and Access in High-poverty Schools (Indicator 4.2). As shown in Table 50, 
slightly more administrators collected data on Staff Training (32 administrators) than on Access in 
High-poverty Schools (29 administrators). Regarding the scope of data collection activities, 29 
administrators collected data from all districts receiving program funds for staff training and 26 
administrators did so for access in high-poverty schools. Three administrators collected data from a 
sample of districts for both indicators. 

Of 35 administrators who collect indicators data, 25 reported using some form of corrective 
action against local school districts that fail to submit indicators data to the state. Specifically, 16 
administrators reported assisting local school districts to write a data collection plan, 12 withheld 
program funds, and four used some other form of corrective action. 

Program administrators most frequently used performance indicators data to report to state 
officials, both for Staff Training (27 administrators) and Access in High-poverty Schools (25 
administrators). Most administrators used data for both indicators to: report to federal officials. 
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Table 48 



Eisenhower Professional Development Program Administrators’ 
Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators listed, how 
is the information used? 


How Data Are Used 


Indicator 

1.1 

(N=25) 


Indicator 

3.2 

(N=29) 


Indicator 

4.1 

(N=22) 


To identify districts and schools for which technical assistance is needed 


14 


16 


13 


To report to state officials 


13 


14 


11 


To report to federal officials 


23 


26 


20 


To identify priorities for state-level technical assistance 


19 


21 


15 


Other 


3 


3 


3 
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Table 49 



Degree of Difficulty Eisenhower Professional Development Program Administrators 
Predict in Collecting Federal Program Performance Indicator Data 



[Where data are NOT collected] If you are not collecting data on any or all of the indicators listed, 
how difficult would it be to start? Please indicate the extent to which data are available from districts 
for each of the indicators listed. 



Level of Difficulty in Collectine Data 


Indicator 

1.1 

(N=2ii 


Indicator 

3.2 

(N=15) 


Indicator 

4.1 

(N=23) 


Could collect from districts and report fairly easily 


1 


2 


4 


Possible to collect from districts and report, but with some difficulty 


11 


10 


14 


Extremely difficult to collect and report 


7 


3 


5 


Not appropriate 


1 


0 


0 


Don’t know 


1 


0 


0 
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Table 50 



Extent to Which TLCF Program Administrators 
Collect Data on Federal Program Performance Indicators 

(N=38) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as to 
the availability of the following information. SEA Currently Collects Data? 


Indicator 


Yes, from 
ALL 
Districts 
receiving 
program 
funds 


SEA Currently Collects Data? 
Yes, from 

a sample Don’t 

of districts No Know 


Other 


3.3: Staff training. Increasing proportions of 
practicing and prospective teachers, school 
administrators, and school librarians will receive 
professional development that enables them to 
effectively use education technology to help students 
learn. 


29 


3 5 


1 


0 


4.2: Access in high-poverty schools. The access to 
education technology in high-poverty schools will be 
comparable to that in other schools. 


26 


3 4 


2 


3 
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identify districts and schools for which technical assistance is needed, and identify priorities for state- 
level technical assistance (Table 51). 

Nine administrators of the Technology Literacy Challenge Fund reported that they do not 
collect data or do not know whether they collect data on at least one program performance indicator. 

Of these, four responded to questions about how difficult it would be to collect these data in the future. 
One program administrator said that she could fairly easily collect data and report on Staff Training, 
and two said that they could possibly collect these data, but with some difficulty. Program 
administrators were slightly less optimistic about their ability to collect data on Access in High-poverty 
Schools: they said that they could possibly collect these data, but with some difficulty (three 
administrators), or that it was not appropriate to collect these data (one administrator). 

Safe and Drug-Free Schools and Communities. The Safe and Drug-Free Schools and 
Communities program had the highest rate of federal program indicators data collection out of all of the 
nine federal programs included in this study. Forty-one out of 48 administrators reported being aware 
of the federal program performance indicators and all 41 administrators reported collecting data on 
Gun-Free Schools Act Notification and Expulsions (Indicator 6. 1); 39 administrators collected data on 
Approval of LEA Applications (Indicator 8.3) (Table 52). 

The Safe and Drug-Free Schools and Communities program also showed the highest numbers 
of administrators reporting taking corrective action when local districts fail to submit program 
performance indicator data. Out of the 41 administrators who collect program performance data, 34 
administrators reported using some form of corrective action against local school districts. Of those, 

23 administrators reported assisting local school districts in writing a data collection plan, 18 withheld 
program funds, and seven indicated that they use some other form of corrective action. 

Safe and Drug-Free Schools and Communities program administrators also seem to target their 
use of performance indicator data, depending on the topic of the indicator. That is, as shown in Table 
53, Gun-Free Schools Act Notification and Expulsions data were used most frequently for reporting to 
federal (39 administrators) and state (31 administrators) officials. By contrast, approval of local 
education agency (LEA) applications (Indicator 8.3) was used most frequently to identify districts and 
schools for which technical assistance is needed (30 administrators), to identify priorities for state-level 
technical assistance, and for federal reporting (29 administrators each). 

Title VI. Administrators of the Title VI program were asked about only one performance 
indicator: Reform Efforts (Indicator 1.1). Of the 48 Title VI program administrators surveyed, 31 
indicated that they are aware of the Department’s program performance indicators. Twenty-four 
administrators indicated that they collect data on the federal performance indicator: 20 said that they 
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Table 51 



TLCF Program Administrators’ 

Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators 
listed, how is the information used? 


How Data Are Used 


Indicator 

3.3 

(N=32) 


Indicator 

4.2 

(N=32) 


To identify districts and schools for which technical assistance is needed 


21 


23 


To report to state officials 


27 


25 


To report to federal officials 


26 


25 


To identify priorities for state-level technical assistance 


20 


20 


Other 


1 


3 
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Table 52 



Extent to Which Administrators for the Safe and Drug-Free 
Schools and Communities Program 
Collect Data on Federal Program Performance Indicators 

(N=41) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as 
to the availability of the following information. SEA Currently Collects Data? 



SEA Currently Collects Data? 





Yes, from 
ALL 
districts 
receiving 


Yes, from a 










program 


sample of 




Don’t 


Other 


Indicator 


funds 


districts 


No 


Know 


6.1: Gun-Free Schools Act notification and 
expulsions. By 1998, all LEAs receiving ESEA 
funds will have policies requiring the expulsion of 
students who bring firearms to school and requiring 
notification of law enforcement. 


41 


0 


0 


0 


0 


8.3: Approval of LEA applications. All states 
will use performance indicators to make decisions 
regarding approval of LEA applications for funding. 
[LEAs are required to have performance indicators 


39 


0 


0 


2 


0 



(called “measurable goals & objectives” in the 
statute) in their applications/plans] 
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Table 53 



Safe and Drug-Free Schools and Communities Program Administrators’ 
Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators 
listed, how is the information used? 


How Data Are Used 


Indicator 

6.1 

(N=41) 


Indicator 

8.3 

(N =39) 


To identify districts and schools for which technical assistance is needed 


16 


30 


To report to state officials 


31 


20 


To report to federal officials 


39 


29 


To identify priorities for state-level technical assistance 


17 


29 


Other 


1 


0 
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collect these data from all districts receiving program funds; four collect data from a sample of districts 
(Table 54). 

Data for Indicator 1.1 do seem to be important to Title VI programs, as two-thirds of program 
administrators, or 14 out of the 21 administrators who responded to the question about corrective 
actions, took some form of corrective action when LEAs failed to submit program performance 
indicator data to them. Of these, eight assisted local school districts in writing a data collection plan, 
five withheld program funds, and three indicated that they used some other form of corrective action. 

Of the 23 program administrators who responded to questions about how they use the indicators 
data, most (17 administrators) said they use the data for the same purposes as other federal programs, 
which are for reporting to federal officials. In addition, 12 administrators report using the indicators 
data to identify priorities for state-level technical assistance; nine use the data to identify districts and 
schools for which technical assistance is needed and to report to state officials; and six use them for 
some other purpose (Table 55). 

For those states that have not collected performance indicator data thus far, collecting it in the 
future seems feasible. Six Title VI program administrators answered questions about how difficult it 
would be to collect these data in the future. Of these, five administrators indicated that it would be 
possible to collect and report these data. Specifically, three indicated that they could collect and report 
these data, but with some difficulty; and two administrators said that they could collect and report these 
data fairly easily. Only one administrator responded that it would be extremely difficult to collect and 
report data on Indicator 1.1. 

Goals 2000. Of the 46 Goals 2000 program administrators surveyed, 16 administrators were 
aware of the Department’s program performance indicators and 14 collected data on them. In 
particular, 13 of the 14 administrators reported collecting data on both indicators about which they 
were being surveyed: Participation in Reform Efforts (Indicator 2.1) and Schools Alignment of Key 
Processes (Indicator 4.2), as shown in Table 56. 

Data on Participation in Reform Efforts was collected from all districts receiving program 
funds more frequently than was data on Schools’ Alignment of Key Processes. The scope of 
administrators’ data collection efforts varied slightly by indicator. That is, 12 administrators collected 
data on Participation in Reform Efforts from all subgrantees, while only eight administrators collected 
data from all subgrantees on Schools’ Alignment of Key Processes. 
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Table 54 



Extent to Which Title VI Program A dminis trator*; 
Collect Data on Federal Program Performance Indicators 

(N=31) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as 
to the availability of the following information. SEA Currently Collects Data? 



SEA Currently Collects Data? 



Indicator 


Yes, from 
ALL 
districts 
receiving 
program 
funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Reform efforts. The use of Title VI funds 
will show evidence that the activities supported are 


20 


4 


4 


2 


0 



integral to achieving district reform plans. 
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Table 55 



Title VI Program Administrators’ 

Use of Federal Program Performance Indicator Data 

(N=23) 



[Where data are collected] If your state collects data on any of the performance 
indicators listed, how is the information used? 



Indicator 



How Data Are Used 1.1 

To identify districts and schools for which technical assistance is needed 9 

To report to state officials 9 

To report to federal officials 17 

To identify priorities for state-level technical assistance 12 

Other 6 
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Table 56 



Extent to Which Goals 2000 Program Administrators 
Collect Data on Federal Program Performance Indicators 

(N=16) 



To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimate as 
to the availability of the following information. SEA Currently Collects Data? 



SEA Currently Collects Data? 



Indicator 


Yes, from 
ALL 
districts 
receiving 
program 
funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


2.1: Participation in reform efforts: By 1999, as 
many as half of the state’s school districts will 
actively participate' in standards-based reform. 


12 


2 


2 


0 


0 


4.2: Schools’ alignment of key processes: 
Surveys of principals and teachers in states with 
standards will indicate that schools have aligned 
curriculum, instruction, professional development 
and assessment to meet challenging state or local 


8 


5 


3 


0 


0 



standards. 




122 

156 



Most Goals 2000 program administrators (11 of 14), reported taking some form of corrective 
action when local education agencies failed to submit program performance indicator data to them. Of 
these, equal numbers of administrators reported assisting local school districts in writing a data 
collection plan and withholding program funds (seven administrators) and one state used some other 
form of corrective action. 

Goals 2000 program administrators seemed to use data on participation in reform efforts 
(indicator 2.1) and schools’ alignment of key processes (indicator 4.2) for the same activities. That is, 
nearly equal numbers of administrators used performance indicator data to report to state officials, 
identify priorities for state-level technical assistance, and identify districts and schools for which 
technical assistance is needed (Table 57). 

Three Goals 2000 program administrators indicated that they do not collect data on at least one 
of the indicators that they were asked about. When asked how difficult it would be in the future to 
collect data on either Participation in Reform Efforts or Schools’ Alignment of Key Processes, one 
administrator said that it is possible to collect these data but with some difficulty . Another 
administrator suggested that it is not appropriate to collect these data. The third administrator said that 
they could easily collect and report data on schools’ alignment of key processes. 



Summary: Program Performance Indicators Arrive 

Both federal and state-developed program performance indicators seem to be taking hold 
among federally funded programs. State-developed performance indicators, for example, are emerging 
in more and more SEAs and about half the state administrators surveyed said they were collecting data 
related to the federal program performance indicators. Nevertheless, there is still room for 
improvement. For example, there is wide variation among programs regarding the extent to which 
indicators data are collected and used. In addition, most administrators who collect federal 
performance indicators data report using the data to report to federal officials rather than to inform 
program performance. 



Table 57 



Goals 2000 Program Administrators’ 

Use of Federal Program Performance Indicator Data 



[Where data are collected] If your state collects data on any of the performance indicators 
listed, how is the information used? 


How Data Are Used 


Indicator 

2.1 

(N=14) 


Indicator 

4.2 

(N=13) 


To identify districts and schools for which technical assistance is needed 


11 


10 


To report to state officials 


13 


11 


To report to federal officials 


9 


7 


To identify priorities for state-level technical assistance 


12 


11 


Other 


0 


1 
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VI. Conclusions 



The policy vision of the Goals 2000: Educate America Act and the reauthorized Elementary 
and Secondary Education Act was one in which schools would strive for significant improvements in 
student performance, using all the resources at their disposal— including the resources they obtained 
under federal programs, many of which continue to provide an extra boost for students at risk of failing 
to meet challenging standards. To help and encourage schools to make such a concerted effort, these 
laws emphasized the opportunities for school districts and state program administrators to break down 
program boundaries and to offer flexibility for local decisions. 

This study provided follow-up information to the baseline study conducted in 1996-97 on the 
early implementation of Goals 2000 and elementary and secondary programs reauthorized under IAS A. 
Both the baseline study and this study were designed to provide data to evaluate the impact of federal 
education programs and to provide data on the use of program performance indicators established 
pursuant to the Government Performance and Results Act. 

This study found that in 1998— four years after reauthorization of ESEA and four-and-a-half 
years after the authorization of Goals 2000— states had made significant progress in implementing the 
legislation in a number of areas. The vast majority of state administrators in 1998 perceived flexibility 
in the legislation, and many reported using that flexibility to help districts find ways to use federal 
program resources to meet local needs, two positive developments since 1996-97. Consistent with 
these findings, more state administrators in 1998 reported requiring consolidated local applications— 
plans describing the intended uses of funds under more than one program — than in 1996-97. Where 
implementation of flexibility provisions fell short in 1998 was in taking advantage of provisions 
allowing consolidation of administrative funds and Ed-Flex provisions: few states reported across-the- 
board consolidation of administrative funds in 1998; most Ed-Flex administrators said that they 
believed that their state and districts were not fully using their Ed-Flex authority. 

There were also some positive changes between 1996-97 and 1998 in state administrators’ 
perceptions of the connections between program purposes and student achievement. In 1998, 
administrators were more likely to recognize such links and to recognize that supporting standards- 
based reform is part of their programs’ purpose. These changes seemed due largely to efforts to 
coordinate program administration and operations across federal programs and the long-awaited 
implementation of state standards and assessment systems. 
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In addition, states appeared to make progress in the area of technical assistance. In 1998, 
program administrators showed greater coordination in the content of the technical assistance they 
provided than in 1996-97, and much of that content focused on a standards-based reform agenda. 
However, as was true in 1996-97, agency downsizing in many SEAs continued to adversely affect the 
technical assistance capacity in federal programs, and interviews with local program administrators 
indicated that states were not meeting their subgrantees’ technical assistance needs. 

As in 1996-97, the area in which implementation of the new federal education legislation most 
fell short of the original federal vision was in accountability for results. Despite progress since the 
baseline study, states still had far to go in building new monitoring procedures that would communicate 
a clear message about the new standards-based accountability framework. Although administrators 
seemed to have access to a great deal of data about student performance and program implementation, 
they were most often using the data for purposes other than judging the success of their programs in 
raising student achievement. 

Thus, we conclude that SEAs have continued to make progress toward implementing the 
federal education legislation. Effects have emerged in a number of areas since the baseline study, 
although areas remain where additional progress is needed. In this regard, it is important to emphasize 
the size of the challenge that the new and reauthorized laws placed before the SEAs in 1994. A great 
deal of organizational learning was called for, and our two studies point to progress not only in 
initiating new administrative routines but also in developing a new outlook on program purposes and 
priorities. Although the administrators of federal programs in SEAs were not uniformly living and 
breathing an agenda of standards-based, data-driven reform in 1998, these studies provide evidence that 
they had moved in that direction in some respects and might be more strongly encouraged and helped 
to do so in other respects. 
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Follow-up Study of State Implementation of 
Federal Elementary and Secondary Education Programs 

CORE SURVEY 



Standards and Assessment: 

SI. Have [this program's] funds helped support the development or review of your state's student performance standards? 



a. Yes (EXPLAIN) 1 

b. No 2 

c. Don't know 3 



S2. [Where assessments aligned with standards have been/are being developed or adopted]: Since 1996, have [this 
program's] funds helped support the development or review of state assessments aligned with standards? 



a. Yes (EXPLAIN) 1 

b. No 2 

c. Don't know 3 



S3. Can you give me examples of any changes in state administrative procedures made by this program because of state 
standards and/or new state assessments? [Probe for 2-3 examples] 



S4. To what extent would you say this program or this state is experiencing the following problems with 
of standards-based reform: (CIRCLE ONE RESPONSE FOR EACH ROW) 



□ No problems. 

Major 

Problem 



a. Standards are not relevant to the services 

that [this program] provides 1 

b. Standards are not specific enough to guide 
teaching and learning (e.g., standards are 

not aligned with the curriculum) 1 

c. Assessments aligned with standards 

are not in place 1 

d. This programs’ state-level staff does not have 

the time to communicate a new program 
purpose driven by state standards 1 

e. This program’s state-level staff does not have 

the expertise to communicate a new program 
purpose driven by state standards 1 

f. Standards have not been approved by ED . . 1 

g. Some districts have standards that are not 

aligned with the state’s standards 1 

h. State standards are changing 1 

i. State assessments are changing 1 

j . Other (SPECIFY) 1 



Moderate Slight 
Problem Problem 



2 3 

2 3 

2 3 



2 3 



2 3 

2 3 



2 

2 

2 

2 



3 

3 

3 

3 



Not a 
Problem 



4 



4 

4 



4 



4 

4 

4 

4 

4 

4 



O 
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the implementation 



Don’t 

Know 



5 



5 

5 



5 



5 

5 

5 

5 

5 

5 
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State Plans: 



Cl. Since January of 1997, has your program conducted specific administrative or operational activities (e.g., monitoring 

local project, providing technical assistance to districts and subgrantees, etc.) in coordination with other federally funded 
education programs? 

a. Yes 1 

b. No (Why not?) . 2 (SKIP to LI) 



C2. Since January of 1997, what among the following administrative or operational activities has your program conducted in 
coordination with other federally funded education programs? (CIRCLE ALL THAT APPLY) 



a. Monitoring local projects 1 

b. Holding local application/planning workshops 1 

c. Providing technical assistance to districts and schools 1 

d. Providing before- or after-school or summer school services 1 

e. Decisionmaking with respect to allocating program resources 

to districts and schools 1 

f. Collecting student performance data 1 

g. Collecting program performance data 1 

h. Conducting local needs assessments 1 

i. Other (SPECIFY) 1 



C3. With which of the following federally funded education programs has your program coordinated the administrative or 
operational activities specified in Question C2? (CIRCLE ALL THAT APPLY) 



a. Title I, Part A 1 

b. Even Start 1 

c. Migrant Education 1 

d. Neglected or Delinquent 1 

e. Eisenhower Professional Development 1 

f. Technology Literacy Challenge Fund 1 

g. Safe and Drug-Free Schools and Communities 1 

h. Title VI 1 

i. Education for Homeless Children and Youth 1 

j. Goals 2000 1 

k. Special Education 1 

l. Other (SPECIFY) . 1 



C4. 



[Where relevant] Why do you coordinate administrative or operational activities with some programs and not with others? 



Subgrant Plans or Applications: 



LI. Does your program require or accept consolidated plans or applications (that is, combined plans or applications for more 
than one program) from local school districts or other subgrantees? (CIRCLE ONE) 



a. Consolidated applications are required 1 

b. Consolidated applications are accepted but not required 2 (SKIP to L4) 

c. No, this program requires a separate application 3 (SKIP to L3) 

d. Consolidated plans or applications are being piloted in some districts 

this year 4 (SKIP to L4) 

e. Other (SPECIFY) 5 (SKIP to L4) 



L2. Why does your program require consolidated plans or applications from local school districts or other subgrantees? 

(NOTE: ANSWER THIS QUESTION AND SKIP TO QUESTION L4) 



L3. Why does your program require a separate subgrant plan or application? (NOTE: ANSWER THIS QUESTION AND 
SKIP TO QUESTION Ml) 



L4. Are reviews of local consolidated plans or applications conducted jointly for [this program] and other programs? 
(CIRCLE ONE) 



a. Yes (SPECIFY Programs) 1 

b. No, applications are not reviewed jointly 2 

c. Other (SPECIFY) 3 

d. Don’t know 4 



L5a. How well would you say the local consolidated plans or applications provide you with the following types of information 
about [this program]? (NOTE: If you do not require districts to report any or all of the following types of information 
in their plans/applications, CIRCLE 5" for NA ) (CIRCLE ONE RESPONSE FOR EACH ROW) 



Very Fairly Not Very 

Well Well Well Poorly NA 



a. Outcome data on program participants .... 1 

b. Community needs assessment 1 

c. Participation and/or attendance data 1 

d. Evaluation of service quality 1 

e. Summaries of services rendered 1 

f. Evidence of additional services needed .... 1 

g. Evidence of varied demographic and ethnic 

group participation 1 

h. Indicators of collaboration with 

other programs 1 

i. Client satisfaction ratings 1 

j. ED’s program performance indicators .... 1 

k. Other (SPECIFY) . 1 



2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 

2 3 4 5 



2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 




L5b. What among the information listed in Question L5a do you need from the consolidated plans/applications to make 
decisions about [this program]? (CIRCLE ALL THAT APPLY) 

a. b. c. d. e. f. g. h. i. j. k. 



L6. What are the strengths and weaknesses of local implementation of the consolidated plans, as far as you know? (NOTE: 
If you don’t know, check the box below and go on to question Ml) (CIRCLE ALL THAT APPLY) 

□ I can’t say; I don’t have enough information about local implementation (GO to Ml) 



a. Working together across programs has been difficult for 

local staff (EXPLAIN) 1 

b. Working together across programs has helped local staff learn about 

each others’ programs and identify opportunities to coordinate 
administrative and operational activities 1 

c. A consolidated plan sharpens local educators’ focus on enabling all 

students to meet the standards 1 

d. A consolidated plan detracts from the attention paid to the purposes 

of [this program] 1 

e. Other (SPECIFY) 1 



How do you know this? 



State Monitoring: 

Ml. In the past 12 months, about what proportion of [this program’s] subgrantees received monitoring visits? 
(CIRCLE ONE) 



a. Half or more 1 

b. Between one-fourth and one -half 2 

c. Fewer than one-fourth 3 

d. None, but we plan to conduct monitoring visits in the future 4 

e. None 5 

f. Other (SPECIFY) 6 



M2. On what basis is it decided which subgrantees to visit? (CIRCLE ALL THAT APPLY) 



a. There is a routine cycle for visits 1 

b. Visits are triggered by information suggesting that the grantee 

is having trouble meeting program requirements (SPECIFY type 

and source of information): 1 

c. Visits are triggered by information about student performance 

(SPECIFY type and source of information): . 1 

d. Subgrantees are visited at their request 1 

e. Other (SPECIFY) 1 



M3. What priorities are addressed through state monitoring? (CIRCLE ALL THAT APPLY) 

a. Student outcomes 1 

b. Progress in tracking activities outlined in local plans 1 

c. Use of federal funds to support state and local reform efforts 1 

d. Compliance with federal programs requirements 1 

e. Other (SPECIFY) 1 



M4. Has this state conducted any integrated monitoring visits that address [this program] and other federal or state programs? 
(CIRCLE ONE) 



a. Yes, with other federally funded programs only 1 

b. Yes, with both federally and state-funded programs 2 

c. No 3 (SKIP to M8b) 

d. Other (SPECIFY) 4 (SKIP to M8b) 



M5. In the past 12 months, approximately what proportion of your program’s monitoring visits were conducted as part of an 
integrated monitoring process? (CIRCLE ONE) 



a. All 1 

b. Half or more, but not all 2 

c. Between one-fourth and one-half 3 

d. Fewer than one-fourth 4 

e. None 5 (SKIP to M8b) 

f. Other (SPECIFY) 6 



M6. With which of the following programs does [this program] participate in integrated monitoring visits? 
(CIRCLE ALL THAT APPLY) 



a. Title I, Part A 1 

b. Even Start 1 

c. Migrant Education 1 

d. N or D 1 

e. Eisenhower (Title II) 1 

f. Technology Literacy Challenge Fund 1 

g. Safe and Drug Free Schools 1 

h. Title VI 1 

i. Education for Homeless Children and Youth 1 

j. Goals 2000 1 

k. Special Education 1 

l. Other programs in the SEA (SPECIFY) . 1 

m. Programs in other state agencies (SPECIFY) . 1 

n. Other (SPECIFY) 1 



O 
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M7. 



[If relevant] To your knowledge, why are certain programs included in integrated monitoring visits while others are not? 



M8a. [For programs participating in integrated monitoring visits] To what extent do you agree with the following statements 
regarding the strengths and weaknesses of your program's project monitoring process? (CIRCLE ONE RESPONSE 
FOR EACH ROW) (NOTE: ANSWER THIS QUESTION AND SKIP TO QUESTION M8c) 



a. 

b. 



c. 

d. 

e. 

f. 
g- 



Strongly 


Somewhat 


Somewhat 


Strongly 


Agree 


Agree 


Disagree 


Disagree 


Integrated monitoring visits have reduced the 

cost of monitoring for this program 


. . i 


2 


3 


4 


Having a variety of programs represented in the 

monitoring process gives a more complete 

picture of what subgrantees are doing 


. . i 


2 


3 


4 


Having a variety of programs represented in the monitoring 
process provides more solutions to subgrantees’ difficulties 


. . i 


2 


3 


4 


Integrated monitoring visits are too general/less 
in-depth than program-specific visits 


. . i 


2 


3 


4 


Integrated monitoring hampers our ability 

to collect the program-specific data we need 


. . i 


2 


3 


4 


Integrated monitoring team members lack the 

expertise to help individual programs 


. . i 


2 


3 


4 


Integrated monitoring teams overwhelm small districts 
where one person may administer several programs 


. . i 


2 


3 


4 



M8b. [For programs that are NOT participating in integrated monitoring visits] To what extent do you agree with the following 
statements regarding the strengths and weaknesses of your program’s project monitoring process? (CIRCLE ONE 
RESPONSE FOR EACH ROW) 

Strongly Somewhat Somewhat Strongly 
Agree Agree Disagree Disagree 



a. We are no longer able to visit as many subgrantees because 

of a lack of staffing/funds (Since when? ) ... 1 2 

b. We are no longer able to visit as many subgrantees because 
of the time demands of revising/developing our monitoring 

process (Since when? ) ... 1 2 

c. Monitoring visits no longer have to focus so strongly on 

compliance issues (Since when? ) ... 1 2 

d. Program staff are knowledgeable about implementation 
and operations issues and can provide technical 

assistance to subgrantees 1 2 



3 4 



3 4 

3 4 



3 4 



M8c. How, if at all, does [this program] provide feedback to local districts about the monitoring visit? (CIRCLE ALL THAT 
APPLY) 



a. By sending districts/subgrantees a written feedback report 1 

b. By addressing the subject at a statewide or regional meeting 1 

c. By giving districts informal feedback over the telephone 1 

d. Other (SPECIFY) 1 

e. It doesn’t (Why not?) . 1 
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M8d. What changes have been made or recommended for state and local program administration and operations as a result of 
[this program’s] monitoring visits/activities? 



Federal Monitoring: 

M9. The U.S. Department of Education (ED) has changed its approach to program monitoring. Are you aware that ED has 
established Regional Service Teams to conduct integrated reviews of federal elementary and secondary education 
programs? 



a. Yes 1 

b. No 2 (SKIP to M12) 



M10. Have you been contacted by a member of a Regional Service Team regarding an integrated review of federal elementary 
and secondary education programs? (CIRCLE ONE) 



a. Yes, our state has been visited by a Regional Service Team 1 

b. Yes, our state has been contacted by a Regional Service Team member. 2 

c. No, our state has had no contact with a Regional Service Team 3 



Mil. How useful do you think the Department’s strategy for conducting integrated reviews will be/has been to implementing 
[this program] to support comprehensive standards-based reform? (CIRCLE ONE) 



a. Very useful . . . 

b. Useful 

c. Somewhat useful 

d. Not useful . . . . 

e. Don’t know . . . 



1 

2 

3 

4 

5 



M12. Do you have comments or concerns regarding integrated program reviews? 



Building Capacity for Improvement: 

PI. In the past year, did any of your written or personal communications encourage subgrantees to pool [this program's] 
funds for professional development with those of any other federal or state program(s)? 



a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 



O 



7 163 ' 



In the past year, has your program funded or directly provided technical assistance to subgrantees on any of the following 
topics: (NOTE: If your program did not fund or directly provide technical assistance to subgrantees, check the box 
below and go on to Question P7) (CIRCLE ALL THAT APPLY) 



□ NA: We did not fund or direcdy provide technical assistance to subgrantees. (GO to P7) 



Directly 
Funded Provided 
IA TA 



Nq 



a. Content or performance standards 1 

b. Student assessment 1 

c. Planning and carrying out whole-school improvement 1 

d. Specific academic subject(s) (e.g., reading, math) 1 

e. Meeting the needs of special populations 1 

f. Adopting and implementing particular model programs 1 

g. Effective roles for instructional aides 1 

h. Techniques for working with families 1 

i. Extended instructional time 1 

j. Use of data-driven decision making (e.g. , student assessments) 1 

k. Use of educational technology 1 

l. Program coordination 1 

m. School safety 1 

n. Other(s) (SPECIFY): . l 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



In the past year, which THREE of the following factors most influenced your selection of technical assistance topics that 
your program funded or directly provided to subgrantees? (CIRCLE THREE ONLY) 



a. Program purposes/goals 1 

b. Goals arising from consolidated planning 1 

c. State goals and priorities 1 

d. Federal legislation/regulations 1 

e. U.S. Department of Education guidance 1 

f. State legislation/regulations 1 

g. Anecdotal information about or personal observations of 

subgrantees’ needs 1 

h. Program evaluation data (SPECIFY) 1 

i. State student assessment data (SPECIFY) 1 

j. Subgrantees’ suggestions 1 

k. Other (SPECIFY) 1 



In your estimation, to what extent has your program been able to meet your subgrantees’ technical assistance needs in the 
past year? (CIRCLE ONE) 



1 (SKIP TO P6) 

2 

3 

4 

Please explain. 



a. Great extent 

b. Some extent 

c. A little . . . 

d. Not at all . . 
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P5. 



What prevents your program from meeting your subgrantees’ technical assistance needs? (CIRCLE ALL THAT 
APPLY) 



a. Insufficient staff size 1 

b. Lack of program funds 1 

c. Lack of knowledge and expertise among state-level staff 1 

d. Lack of knowledge and expertise among available contractors 1 

e. Other (SPECIFY) 1 



P6. What are the priorities for allocating this program’s technical assistance resources? (CIRCLE ALL THAT APPLY) 



a. Districts with low achievement 1 

b. Districts with less experienced program managers 1 

c. Districts that request help 1 

d. High-poverty districts 1 

e. Districts with program compliance problems 1 

f. Other (SPECIFY): 1 



P7. In the past year, how helpful have each of the following sources of information been in informing your understanding of 
the federal legislative provisions affecting your program? (CIRCLE ALL THAT APPLY) 







Very 




A Little 


Not at All 


No 






Helpful 


Helpful 


Helpful 


Heipftq 


Contact 


a. 


Written information from U.S. Department 
of Education (ED) (e.g., guidance, other mailings) 


. . 1 


2 


3 


4 


5 


b. 


Other contacts with ED (e.g., conferences, 
workshops, on-line services, telephone) 


. . 1 


2 


3 


4 


5 


c. 


Regional Educational Laboratories 


. . 1 


2 


3 


4 


5 


d. 


Comprehensive Centers 


. . 1 


2 


3 


4 


5 


e. 


Eisenhower Math/Science consortia 


. . 1 


2 


3 


4 


5 


f. 


Professional associations (e.g., CCSSO) 


. . 1 


2 


3 


4 


5 


g- 


Institutions of higher education 


. . 1 


2 


3 


4 


5 


h. 


Other states 


. . 1 


2 


3 


4 


5 


i. 


Other (SPECIFY) 


1 


2 


3 


4 


5 



P8. In the past year, how would you rate the timeliness with which you have received information from the U.S. Department 
of Education regarding the federal legislative provisions affecting your program? (CIRCLE ONE) 



a. Very slow 1 

b. Somewhat slow 2 

c. Somewhat timely 3 

d. Very Timely 4 
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P9. [Where applicable] With respect to each of the following topics, how helpful was the technical assistance 

provided by the [INTERVIEWER: Write in name of Center] in the 

past year? (CIRCLE ONE RESPONSE FOR EACH ROW) 



□ NA: We have had no contact with the Comprehensive Center serving 
our region/program (GO TO P10) 



NA: Didn’t Very A Little Not atAll 

Receive Helpful Helpful Helpful Helpful 



a. Content or performance standards 1 

b. Student assessment 1 

c. Disaggregation of performance data 1 

d. Designing assessments to accommodate the needs of 
students with limited English proficiency and students 

with Individual Educational Programs (IEPs) 1 

e. Developing and implementing measures of adequate 

yearly progress (AYP) 1 

f. Identifying schools in need of improvement 1 

g. Developing and reviewing applications for the 

Comprehensive School Reform Demonstration Program ... 1 

h. Planning and carrying out whole-school reform 1 

i. Specific academic subject(s) (e.g., reading, math) 1 

j. Meeting the needs of special populations 1 

k. Other (SPECIFY) 1 




4 5 

4 5 

4 5 



4 5 

4 5 

4 5 

4 5 

4 5 

4 5 

4 5 

4 5 



P10. When you are making decisions about how to administer [this program], which of the following sources of guidance are 
generally the most influential? Which THREE of the following are the most influential sources of guidance? (CIRCLE 
THREE) 



a. U.S. Department of Education 1 

b. Key policymakers in the SEA 1 

c. Other state administrators of federal programs in this state 1 

d. State administrators of federal programs in other states 1 

e. Multi-state providers of technical assistance (e.g.. Comprehensive 

Regional Assistance Centers) 1 

f. Local districts or other subgrantees 1 

g. Other (SPECIFY) 1 



Educational Technology: 

Tl. Have [this program’s] funds helped support the development of a statewide plan for acquiring and using technology in 
education? 



a. Yes 1 

b. No 2 
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T2. In the past year, has the statewide plan for acquiring and using technology in education 

influenced [this program’s] decisionmaking with respect to educational technology purchases? 



a. Yes (EXPLAIN) 1 

b. No 2 

c. Not familiar with statewide plan for acquiring and using technology 

in education 3 



T3. In the past year, to what extent has the Technology Literacy Challenge Fund program influenced 

[this program’s] decisionmaking with respect to educational technology purchases? (CIRCLE ONE) 



a. To a great extent 1 

b. To some extent 2 

c. A little 3 

d. Not at all 4 



Please explain. 



T4. Have [this program’s] funds been used for educational technology purchases? 



a. Yes 1 

b. No 2 



Administrative Flexibility: 



FI. Is your state using the provision in the reauthorized ESEA that allows it to consolidate administrative 
funding under different programs? (CIRCLE ONE) 



a. Yes 1 

b. Yes, but [this program] is not part of the consolidation 2 (SKIP to F5) 

b. No (Why not?) . 3 (SKIP to F5) 

c. Don't know 4 (SKIP to F5) 



F2. With which of the following programs is your program consolidating its administrative funds? (CIRCLE ALL THAT 
APPLY) 



a. Title I, Part A 1 

b. Even Start 1 

c. Migrant Education 1 

d. Neglected or Delinquent 1 

e. Eisenhower Professional Development 1 

f. Technology Literacy Challenge Fund 1 

g. Safe and Drug-Free Schools and Communities 1 

h. Title VI 1 

i. Education for Homeless Children and Youth 1 

j. Goals 2000 1 

k. Other (SPECIFY) 1 



O 
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F3. 



To what extent has this consolidation of funds affected the way you do your job? How? 



a. To a considerable extent (SPECIFY) ' . 1 

b. Somewhat (SPECIFY) . 2 

c. Not at all (PLEASE EXPLAIN) . 3 

d. [Program] not part of the consolidation of administrative funds 4 (SKIP to F5) 



F4. To what extent has consolidating administrative funds affected your program’s capacity to do the following? (CIRCLE 
ONE RESPONSE FOR EACH ROW) 







Increased 


Decreased 


No 


Don’t 








Caoacitv 


Capacity 


Effect 


Know 


N/A 


a. 


Provide technical assistance to subgrantees 


i 


2 


3 


4 


5 


b. 


Participate in state-level cross-program planning .... 


i 


2 


3 


4 


5 


c. 


Coordinate program services and operations among 
other state and federal programs 


i 


2 . 


3 


4 


5 


d. 


Monitor local projects 


i 


2 


3 


4 


5 


e. 


Participate in the development of state standards .... 


i 


2 


3 


4 


5 


f. 


Participate in the development of state assessments . . . 


i 


2 


3 


4 


5 




Accept and review consolidated subgrant applications . 


i 


2 


3 


4 


5 


h. 


Engage in data-driven decisionmaking 


i 


2 


3 


4 


5 


i. 


Other (SPECIFY) 


i 


2 


3 


4 


5 



F5. Taking into account all of your office’s responsibilities under this program, to what extent do you find that this legislation 
gives you more administrative flexibility than you had before the reauthorization of the Elementary and Secondary 
Education Act of 1994? (NOTE: IF TLCF PROGRAM, SKIP to QUESTION F6) (CIRCLE ONE) 



a. To a considerable extent 1 

b. Somewhat 2 

c. Not at all 3 

d. No change 4 



F6. Please elaborate on the ways in which you need additional flexibility from the federal and state levels to more 
successfully administer your program. 



F7. 



O 




In what ways, if at all, has flexibility from the federal and state levels-or lack of flexibility-affected the 
performance of your program? 



Program Accountability: 



Al. In general, how often does this program collect and/or receive program performance information from local 
school districts, schools, or subgrantees? (CIRCLE ONE) 



a. Quarterly 1 

b. Semiannually 2 

c. Annually 3 

d. Every two years 4 

e. Every three years 5 

A2. Does this program collect any information related to student performance? 

a Yes 1 

b. No 2 (SKIP to A4) 



A3a. What information about student performance are local school districts, schools, and subgrantees required to report to this 
program regarding program participants? (NOTE: If categories of student information are not available for students 
participating in this program, CIRCLE “3" for “NA”) (CIRCLE ONE RESPONSE FOR EACH ROW) 



□ 



a. 

b. 

c. 

d. 

e. 

f. 

g- 

h. 

i. 

j- 

k. 



This program does not require school districts, schools, or subgrantees to report on program participants’ 
performance (GO to A4). 



Performance results from the state assessment 

Performance results from tests other than the state’s assessment . . 
Performance results from course-related tests (e.g., end-of-course, 

advanced placement, state honors examinations) 

Distribution of student grades 

Rates of student participation in accelerated courses or academic 

honors programs in high poverty schools 

Examples of student work 

Rates of participation in extracurricular activities 

Dropout rates 

Absentee rates 

Incidences of school disruption 

Other (SPECIFY) 



Yes No NA 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 
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Of the student performance information that local school districts, schools, and subgrantees are required to report to this 
program, which, if any, must be disaggregated or broken down in some way-for example, by school, poverty level, 
race/ethnicity, migrant status, or limited English proficiency (LEP) status? (NOTE: If categories of information are not 
available for students participating in this program OR are not disaggregated in some way, CIRCLE “N/A”) 

Student Performance Information is Disaggregated by: 









Poverty 


Race/ 


Migrant 


LEP 








School 


Level 


Ethnicity 


Status 


Status 


N/A 


a. 


Performance results from the state assessment 


i 


2 


3 


4 


5 


6 


b. 


Performance results from tests other than the state’s 
















assessment 


i 


2 


3 


4 


5 


6 


c. 


Performance results from course-related tests (e.g., end-of- 
















course, advanced placement, state honors examinations) . . 


i 


2 


3 


4 


5 


6 


d. 


Distribution of student grades 


i 


2 


3 


4 


5 


6 


e. 


Rates of student participation in accelerated courses or 
















academic honors programs in high poverty schools 


i 


2 


3 


4 


5 


6 


f. 


Examples of student work 


i 


2 


3 


4 


5 


6 


g- 


Rates of participation in extracurricular activities 


i 


2 


3 


4 


5 


6 


h. 


Dropout rates 


i 


2 


3 


4 


5 


6 


i. 


Absentee rates 


i 


2 


3 


4 


5 


6 


j- 


Incidences of school disruption 


i 


2 


3 


4 


5 


6 


k. 


Other (SPECIFY) 


i 


2 


3 


4 


5 


6 



A4. What information about program implementation are local school districts, schools, or other subgrantees required to 

report to this program? (NOTE: If categories of program implementation are not applicable or relevant to this program, 
CIRCLE “3" for “NA”) (CIRCLE ONE RESPONSE FOR EACH ROW) 

Yes No NA 



a. Summaries of services rendered 

b. Evaluations of service quality 

c. Client satisfaction ratings 

d. Evidence of additional services needed 

e. Levels of parent involvement 

f. Levels of community involvement (other than parents) 

g. Evidence of varied demographic and ethnic group participation 

h. Indicators of collaboration with other programs 

i. Staff satisfaction ratings 

j. Other (SPECIFY) 



1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 



A5. 



How does this program ensure that appropriate measures are being followed among local school districts, schools, or 
other subgrantees in processing and reporting student performance and program implementation data? 



In addition to the information that local school districts, schools, or other subgrantees may report to this program, what 
data collection strategies does this program use to obtain program performance information ? (CIRCLE ALL THAT 
APPLY) 

a. Mail surveys (e.g., of program staff, of clients, etc.) 1 

b. Telephone surveys 1 

c. Surveys of project administrators (e.g. , distributed and returned at 

program-sponsored events such as workshops or conferences) 1 

d. On-site observations of local project services and activities 1 

e. In-person interviews of project administrators 1 

f. Customer focus groups/surveys 1 

g. Analyses of requests for information from the field 1 

h. Web site “hits” 1 

i. Informal conversations with project staff 1 

j. We use data from other offices within the SEA (e.g., assessment) 

(SPECIFY TYPE and SOURCE of DATA) 1 

k. Other (SPECIFY) 1 

l. None of the above 1 

How does this program use the student performance and program implementation information it collects from the field? 
(CIRCLE ALL THAT APPLY) 

Student Program 

Performance Implementation 



a. To identify program services that need to be extended or reduced 

b. To assess progress of underserved ethnic or demographic groups 

c. To report to managers within the state agency 

d. To report to the state board of education 

e. To report to local school districts, schools, and subgrantees . . . 

f. To report to the federal government 

g. To identify districts in need of improvement 

h. To identify schools in need of improvement 

i. Other (SPECIFY) 



1 

1 

1 

1 

1 

1 

1 

1 

1 



2 

2 

2 

2 

2 

2 

2 

2 

2 



Which, if any, of the following have occurred as a result of the availability of program and student performance data, as 
described above? (CIRCLE ALL THAT APPLY) 



a. Improved communication with program subgrantees 1 

b. Provided documentation to support funding requests 1 

c. Enabled us to quickly address the performance variability that occurs 

among demographic and ethnic groups 1 

d. Enabled us to adjust programming 1 

e. Focused the program staff on student results and achievement 1 

f. Promoted coordination with other federal, state, and local programs 1 

g. Enabled program staff to inform the state board of education and other 

interested public communities about program status 1 

h. Other (SPECIFY) 1 

Outside of Title I school improvement, does this state have its own procedures for identifying failing schools and 
correcting their problems? (CIRCLE ONE) 



a. Yes 

b. No 

c. Don’t know 
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1 

2 (SKIP to Ol) 

3 (SKIP to Ol) 



A 10. Does your program make any decisions or provide any services based on state-level data identifying failing schools? 

a. Yes (SPECIFY) . { 

b. No (WHY NOT?) 2 



Not yet; in the process of developing an accountability system that makes 

use of state-level data identifying failing schools 3 



Overall Successes and Problems: 

01 . Taking into account your experience with the administration of [this program] in the last 12 months, 
what would you say is working well? How do you know this? Which legislative provisions have 
contributed to these successes? 



What would you say have been the greatest problems in implementation in the last 12 months? 
How do you know this? Which legislative provisions have contributed to these problems or 
are barriers to [this program] ’s success? 



What about your program’s organization and/or operations affects the extent to which you can focus 
on improving student achievement? 



04. 



In what areas do you think your state has the farthest to go in meeting its own reform goals? 



In what ways, if any, do you think the reauthorized [program] reinforces the direction in which 
your state is moving? 
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06. In your estimation, are your program funds used to support state reform goals? 

a. Yes (SPECIFY) 1 

b. No . 2 

07. Since the reauthorization of ESEA in 1994, has your SEA been downsized or reorganized in a way 
that has affected the staffing of [this program]? 

Yes (SPECIFY) 1 

No 2 

if at all, does your SEA’s organization and operations affect the administration of your program? 



a. 

b. 

08. How, 




ii73 



2a. 



2b. 



3. 



Title 1/ 
O 




Additional Items for the State Coordinator of Title I, Part A 



What has your office done to inform districts and schools about the provisions for schoolwide 
programs? (CIRCLE ALL THAT APPLY) 



a. Conducted workshops that discuss schoolwide programs among other topics . . 1 

b. Conducted workshops specifically focused on schoolwide programs 1 

c. Called district officials to suggest they consider schoolwide programs 1 

d. Sent information by mail or e-mail to district officials 1 

e. Sent information by mail or e-mail to principals of eligible schools 1 

f. Other (SPECIFY) 1 

g. None of the above 1 



What percent of the following types of schools did school support teams serve in the past 12 months? (NOTE: If 
you do not know, CIRCLE “1" for “Don’t Know”) 

Percent of Don’t 

Schools Know 

a. Schools in need of improvement % 1 

b. Schoolwide programs % 1 — 

c. Other (SPECIFY) % 1 — 



In the past 12 months, were there more schools in need of school support team services than your program could 
accommodate? 

a. Yes 1 

b. No, my program was able to accommodate all schools that had a need 2 

c. Other (SPECIFY) 3 



Among the following, select the FIVE topics on which schools and districts most frequently request assistance from 
school support teams. (CHOOSE FIVE) 



a. I am not familiar enough with the team’s activities to know 1 

b. Strategies for conducting needs assessments 1 

c. Using assessment information from multiple data sources 1 

d. Collecting and analyzing non-cognitive student data (i.e., attendance, 

participation in extra curricular activities, support for learning at home) 1 

e. Understanding cultural diversity 1 

f. Developing curriculum 1 

g. Understanding and implementing standards-based reforms 1 

h. Identifying successful instructional strategies/improving instruction 1 

i. Evaluating school wide programs 1 

j. Identifying additional sources of program funding 1 

k. Adopting/implementing particular model programs 1 

l. Other (SPECIFY) 1 



(State) 



1 
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Tide 1/ 

O 




What kind of a commitment to schools do school support teams typically make? (CIRCLE ONE) 



a. Single event 1 

b. Provide a series of structured, pre-defined events 1 

c. Level of commitment depends on the nature of the school’s or LEA’s request . 1 

d. Guarantee a minimum number of consultations for one year 1 

e. Guarantee consultation for more than one year (SPECIFY ft YEARS ) 1 

f. Other (SPECIFY) \ 



How does this state evaluate the effectiveness of its school support teams? (CIRCLE ALL THAT APPLY) 



a. Systematic data collection from participating schools ; . 1 

b. More informal feedback from participating schools 1 

c. Feedback from school support teams 1 

d. Other (SPECIFY) \ 

e. It doesn’t \ 



Has your office issued written guidance to school districts regarding the provision of Title I services to private 
school students based on the Agostini v. Felton decision? 

a. Yes (WHEN?) , 

b. No 9 



(State) 2 
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7. Among the following Title I assessment requirements, which have been difficult for this state to understand or to 
carry out? (NOTE: If none, check the box below and go on to Question 8a) (CIRCLE ALL THAT APPLY) 

□ None (GO TO QUESTION 8a) 



a. Developing or adopting assessments that are aligned with the state’s 
challenging content and student performance standards and provide 

coherent information about student attainment of such standards 1 

b. Ensuring that assessments are administered at least once between .... 

grades 3-5, and again between grades 6-9 and grades 10-12 1 

c. Developing or adopting assessments that are consistent with nationally 

recognized professional and technical standards 1 

d. Developing or adopting assessments that measure student proficiency 

in the academic subjects where there are state content standards 1 

e. Developing or adopting assessments that measure student proficiency in the 

academic subjects where there are student performance standards 1 

f. Developing or adopting assessments that provide for reasonable adaptations and 

accommodations for students with special education needs 1 

g. Developing or adopting assessments that provide for 

reasonable adaptations and accommodations for migrant students . . 1 

h. Developing or adopting assessments that provide for reasonable adaptations and 

accommodations for students with limited English proficiency 1 

i. Developing or adopting assessments that allow for the disaggregation of results within 

each state, district, and school by gender, race, ethnicity, English proficiency, and 
migrant status (SPECIFY AREAS OF DIFFICULTY) ' 1 

j. Tracking academic progress of students who transfer among 

schools within a single academic year 1 

k. Developing or adopting assessments that enable comparisons to be 

made between economically disadvantaged/advantaged students 1 

l. Responding to the Title I requirements in a state where districts 

choose their own assessments 1 

m. Other (SPECIFY) ’’’ 1 



For this year, 1997-98, is this state using transitional assessments for Title I? (NOTE: In the space provided, the 
interviewer should write the information listed in the state plan and review it with the respondent) 



a. Yes (DESCRIBE) 1 

b. No; the state allows districts to choose their own assessments 2 (SKIP to Q12a) 

c. No; the state’s assessment system has been approved by the U.S. Department 

of Education 3 (SKIP to Q13) _ 

Are the transitional assessments used for Title I accountability the same assessments as those used for the state? 

a. Yes 1 

b. No (EXPLAIN) * 2 

c. Don’t know 3 



Title 1/ 




(State) 
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To what extent, if at all, are special accommodations made in the transitional assessment to include the following 
categories of students? (CIRCLE ONE RESPONSE FOR EACH ROW) 







Great 


Some 


A 


Not at 


Don’t 






Extent 


Extent 


Little 


All 


Know 


a. 


Students with limited English proficiency . . . . 


1 


2 


3 


4 


5 


b. 


Migrant students 




2 


3 


4 


5 


c. 


Students with disabilities 


1 


2 


3 


4 


5 


d. 


Other (SPECIFY) 


1 


2 


3 


4 


5 



What percentage of students in the following categories are included in Title I transitional assessments? 
(NOTE: If you don’t know, circle “1" under “Don’t Know”) 



Percent of Students Included Don’t 
In Transitional Assessment Know 



a. Students with limited English proficiency 1 

b. Migrant students 1 

c. Students with disabilities 1 

d. Other (SPECIFY) . 1 



[If using transitional assessment]. How do you anticipate that the final assessment will differ from the 
current transitional assessment? (NOTE: ANSWER THIS QUESTION AND GO TO QUESTION 15) 



Please describe the tests and procedures, if any, districts use for Title I accountability. 



In your estimation, to what extent are districts carrying out the Title I requirements for assessment? 
(CIRCLE ONE) (NOTE: ANSWER THIS QUESTION AND GO ON TO QUESTION 15) 



a. To a great extent 1 

b. Somewhat 2 

c. A little 3 

d. Not at all 4 



How do you know this? 



Please describe the tests and procedures currently used for Tide I in your state [NOTE to Interviewer: In the space 
provided, write the Title I tests and procedures as described in the state plan; verify that the information is current.] 



1/ 



(State) 



4 
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To what extent, if at all, are the following categories of students included in the tests and procedures 
used for Title I? 

Great Some A Not at Don’t 
Extent Extent Little All Know 



a. Students with limited English proficiency . 1 

b. Migrant students 1 

c. Students with disabilities 1 

d. Other (SPECIFY) 1 



2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 



[NOTE to Interviewer: Review the definition of adequate yearly progress (AYP) for schools and local school districts in the 
state plan. Write the definitions of AYP for schools and for school districts in the spaces below and review them with the 
respondent. Verify that definitions still apply.] 

15. [Review measure of adequate yearly progress for schools and local school districts! 

a. (DESCRIBE AYP for SCHOOLS) 1 



b. (DESCRIBE AYP for DISTRICTS) 1 



c. The state has not yet developed measures of adequate yearly progress 

for either schools or local school districts (WHY NOT?) 1 (SKIP to QI7a) 



16a. If your state has developed adequate yearly progress measures for schools and/or local school districts, do you 

personally believe the expectations are too high, too low, or about right (CIRCLE ONE RESPONSE FOR EACH 
ROW) 





Too 


Too 


About 


Don’t Have a 




High 


Low 


Right 


Measure Yet 


Schools 


1 


2 


3 


4 


Local school districts 


1 


2 


3 


4 



16b. Is the Title I accountability system the same as the accountability system used for the state? (CIRCLE ONE) 

a. Yes 1 

b. No; the state does not have an accountability system 2 

c. No 3 

d. Don’t know 4 



Title I/_ 

ERIC " 



(State) 
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17a. ' 



17b. 



18. 



19. 



Tide 1/ 

O 
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What challenges, if any, do you face in identifying Title I districts in need of improvement? (CIRCLE ALL THAT 
APPLY) 



a. Applying the measure(s) of progress included in the definition 

of AYP for districts 1 

b. Measuring growth when the state assessment system keeps changing 1 

c. Measuring growth when the local assessment system keeps changing 1 

d. Conveying to districts the criteria that are used in identifying districts 

in need of improvement • 1 

e Collecting school-level data from districts 1 

f. Accessing state-level student performance data 1 

g. Other (SPECIFY) 1 

h. None 1 



What challenges, if any, do districts face in identifying Title I schools in need of improvement? (CIRCLE ALL 
THAT APPLY) 



a. Applying the measure(s) of progress included in the definition 

of AYP for schools 1 

b. Measuring growth when the state assessment system keeps changing 1 

c. Measuring growth when the local assessment system keeps changing 1 

d. Conveying to schools the criteria that are used in identifying schools 

in need of improvement 1 

e. Collecting school-level data from districts 1 

f. Accessing state-level student performance data 1 

g. Other (SPECIFY) 1 

h. None 1 



Does the state reward local school districts-at some time interval (e.g., annually, biannually)-that have the best record 



of improving student performance? 

a. Yes [SPECIFY REWARD(S)] 1 

b. No 2 



In which of the following ways does this state initiate corrective action against LOCAL SCHOOL DISTRICTS whose 
schools have not made adequate progress in improving student performance? (CIRCLE ALL THAT APPLY) 



a. It does not 1 

b. Assist local school districts in developing a plan to address 

specific student problems 1 

c. Withhold Title I funds 1 

d. Reconstitute school district personnel 1 

e. Remove particular schools from the local school district’s jurisdiction and 
establish alternative arrangements for their public governance and supervision . 1 

f. Abolish or restructure the local school district 1 

g. Authorize students to transfer from low performing schools 1 

h. Other (SPECIFY) . 1 



(State) 6 
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In which of the following ways does this state reward SCHOOLS that have the best record of improving student 
performance? (CIRCLE ALL THAT APPLY) 



a. It does not 1 

b. Designate schools as distinguished 1 

c. Encourage educators in rewarded schools to serve as mentors 

to other educators and schools 1 

d. Include educators from achieving schools on school support teams 1 

e. Provide monetary rewards to achieving school districts 1 

f. Provide monetary rewards directly to the staff in achieving schools 1 

g. Other (SPECIFY) 1 



Using the U.S. Department of Education definition of performance indicators as measures that show the degree to 
which key programs achieve desired performance levels, has this program developed performance indicators? 



a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 3 



Has your program jointly developed performance indicators with other federal programs? 



a. Yes (SPECIFY PROGRAMS) . 1 

b. No 2 



If your program has developed program performance indicators, how do you and your colleagues in this program 
office use them? (CIRCLE ALL THAT APPLY) 



a. To provide state and local administrators with a common 

language for defining program results 1 

b. To help program staff focus on program goals and objectives 1 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 

f. To provide information about existing or potential problems 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 

h. To provide consumer-oriented information about program performance 1 

i. Other (SPECIFY) 1 



In response to the requirements of the Government Performance and Results Act, the U.S. Department of Education 
(ED) has developed a set of performance indicators for the Title I Part A program. Are you aware of the indicators? 

a. Yes 1 

b. No 2 (SKIP to Q31) 



1 / 



(State) 
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To assist in updating and improving ED’s performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability of 
the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 



Performance 

Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 
districts 
receiving 
program funds 


Yes, from 
a sample 
of districts 


No 


Don’t 

Know 


Other 


2.3: Research-based curriculum and instruction. The 
proportion of schools using comprehensive, research-based 
approaches to improve curriculum and instruction. 


1 


2 


3 


4 


5 


2.6: Qualified teacher aides. The percent of districts 
providing support for the educational improvement of 
paraprofessionals/ teacher aides will increase 


1 


2 


3 


4 


5 


4.3a: Accountability: intervention. States and districts 
provide assistance to schools not making progress (through 
school support teams and other sources). 


1 


2 


3 


4 


5 


4.3b: Accountability: assistance. States and districts will 
take appropriate action with schools that consistently fail. 


1 


2 


3 


4 


5 



[Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state on 
the performance indicators listed in Question 25? 



[Where data are collected] If your state collects data on any of the performance indicators (listed above), how is the information 
used? (CIRCLE ALL THAT APPLY) 



a. 

b. 

c. 

d. 

e. 



Indicator 

2J 

To identify districts and schools for which technical assistance 



is needed 1 

To report to state officials 1 

To report to federal officials 1 

To identify priorities for state-level technical assistance 1 

Other (SPECIFY) 1 



Indicator Indicator Indicator 
2.6 4.3a 4.3b 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 
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28 . 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 

[NOTE: ANSWER THIS QUESTION AND GO ON TO QUESTION 31] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) . . 1 



29a. [Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 25, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 25. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for 
Not Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 





Could Collect 


Possible to Collect 


Extremely 








from Districts 


from Districts & 


Difficult to 








and Report 


Report, But w / 


Collect & 


Not 


Don’t 




Fairlv Easilv 


Some Difficulty 


Report 


Appropriate 


Know 


a. 


Indicator 2.3: Research-based curriculum and 
instruction 1 


2 


3 


4 


5 


b. 


Indicator 2.6: Qualified teacher aides 1 


2 


3 


4 


5 


c. 


Indicator 4.3a: Accountability: intervention . 1 


2 


3 


4 


5 


d. 


Indicator 4.3b: Accountability: assistance . 1 


2 


3 


4 


5 


29b. 


[Where relevant] Why do you consider it inappropriate to 


collect data on some 


or all of the indicators listed 


in Question 



25? 



29c. [Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 25? 



30. Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 25? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 



Title 1/ (State) 

O 
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31. 



Does or will this office administer the Comprehensive School Reform Demonstration Program? 



a. Yes 

b. No 

c. Don’t know 



1 (SKIP to Q33) 

2 

3 (SKIP to Q37) 



32. What office administers the Comprehensive School Reform Demonstration Program (CSRDP)? (NOTE: If you do not 
know, please say so and GO TO QUESTION 37) 

Office/Division/Department: 

Contact Person: 

Telephone Number: 

NOTE: Interviewer should contact the office/division/department administering the CSRDP and ask the following 
questions. 



33. Please rate the difficulty of implementing each of the following new provisions for the Comprehensive School Reform 
Demonstration Program in this state: (CIRCLE ONE RESPONSE FOR EACH ROW) 



a. 

b. 



c. 

d. 



e. 

f. 



Not at All Minor Moderate Very 
Difficult Difficulty Difficulty Difficult 



Developing a process and selection criteria 

for awarding competitive grants to LEAs 1 

Ensuring that only high-quality, well-defined, 
well-documented comprehensive school reform 

programs are funded 1 

Obtaining and disseminating materials identifying 
research-based comprehensive school 

reform models . 1 

Providing technical assistance to -local school 
districts and schools in evaluating, selecting, 
developing, and implementing comprehensive 

school reforms 1 

Planning for the evaluation of reform implementation 

and the measurement of student results 1 

Other (SPECIFY) 1 



2 3 4 

2 3 4 

2 3 4 



2 3 4 

2 3 4 

2 3 4 



Title II (State) 
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34 . 



How helpful have each of the following sources of information been in informing your understanding of the 
Comprehensive School Reform Demonstration (CSRD) program? (CIRCLE ONE RESPONSE FOR EACH ROW) 



a. 

b. 

c. 

d. 

e. 

f. 

fi- 

ll. 

i. 

j- 

k. 



Very 

Helpful Helpful 



Written information from U.S. Department 

of Education (ED) (e.g., guidance, other mailings) . . 1 2 

Other contacts with ED (e.g., conferences, 

workshops, on-line services, telephone) 1 2 

Regional Educational Laboratories 1 2 

Comprehensive Centers 1 2 

Eisenhower Math/Science Consortia 1 2 

Professional associations (e.g., CCSSO) 1 2 

Education periodicals/publications 1 2 

Institutions of higher education 1 2 

Other states 1 2 

Other private non-governmental organizations 

or foundations 1 2 

Other (SPECIFY) . 1 2 



A Little Not at All No 
Helpful Helpful Contact 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 



35. How has your office informed local districts and schools about the CSRD? (CIRCLE ALL THAT APPLY) 



a. By forwarding copies of the U.S. Department of Education's guidance 1 

b. By sending out written information developed by the state (GET A COPY) 1 

c. By addressing the subject at statewide or regional meetings 1 

d. By providing technical assistance to districts in the process of preparing 

CSRDP applications 1 

e. By other means (SPECIFY) 1 



36. What kind of assistance, if any, has this office provided to districts that have received or are considering applying for 
subgrants under the CSRD? (CIRCLE ALL THAT APPLY) 



a. Send written guidance developed by the state on selecting a 

research-based comprehensive school reform model 1 

b. Provide workshops on selecting a research-based comprehensive school reform model . . 1 

c. Host statewide showcases on selecting a research-based 

comprehensive school reform model 1 

d. Organize meetings with model developers, districts, and schools 1 

e. Address the subject at statewide or regional meetings 1 

f. Other (SPECIFY) 



1 



1/ (State) 

ERIC 
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37. 



Does this state have a special policy emphasis in reading? 



1 

2 (GO to Q39) 



a. Yes 

b. No 

38. What is the role of Title I in supporting your state’s special policy emphasis in reading? What are some examples? 



39. Is Title I coordinated with special education at the state level to promote reading? 



a. Yes (Please explain) 1 

b. No 2 




(State) 



191 



12 



Additional Items for the State Even Start Coordinator 



1. How many local subgrant awards did this state make over the past 12 months? 

2. What was the total number of applications received for subgrant awards? 

3. From which of the following sources are data used to determine local need for an Even 



Start subgrant? (CIRCLE ALL THAT APPLY) 

a. Applicant districts 1 

b. Other SEA program offices (SPECIFY) . 1 

c. Other state agencies (SPECIFY) 1 

d. Other (SPECIFY) . 1 



4. In written communication-or in workshops, monitoring, or other interactions-what did you do 
to help districts fulfill their funding contribution requirement? 



5. 



Do any of the following programs provide supplementary funding to local Even Start subgrant projects? (CIRCLE 
ALL THAT APPLY) 



a. Title I 1 

b. Head Start 1 

c. Adult Education 1 

d. Other (SPECIFY) . 1 



e. No 



1 (SKIP to Q7) 



6. [If supplementary funding is provided] Does this represent a change from what has been 



done in the past? 

a. Yes (EXPLAIN) . 1 

b. No 2 



Even Start/ 



ERIC 



(State) 



1 
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7. What factors have been major barriers to your efforts to collaborate with the following programs, agencies, and 



organizations? (CIRCLE ALL THAT APPLY) 

Policies, 

Regulations, 

Laws 


Time 


Culture of 
Organization/ 
Differences in 
Program 
Philosophy 


Differences 
in Goals and 
Objectives 


Other 


a. Title I 


i 


2 


3 


4 


5 


b. Adult Education 

c. Head Start State Collaboration 


i 


2 


3 


4 


5 


Grantees 


i 


2 


3 


4 


5 


d. State Welfare Office 


i 


2 


3 


4 


5 


e. State Department of Labor . . . 

f. Other programs within 


i 


2 


3 


4 


5 


the state (SPECIFY) 


i 


2 


3 


4 


5 


h. Goals 2000 


i 


2 


3 


4 


5 


i. State Literacy Councils 


i 


2 


3 


4 


5 


j. State Library Councils 

k. State Parents as Teachers 


i 


2 


3 


4 


5 


Association 


i 


2 


3 


4 


5 


1. Other (SPECIFY) 1 

How does the state measure success in Even Start local 


2 

programs? 


3 


4 


5 



a. Number of families served 1 

b. Number of project collaborators 1 

c. Adult gains on measures of adult literacy or English proficiency 1 

d. Number of adults receiving a GED 1 

e. Adult gains in parenting knowledge or skills 1 

f. Child gains on measures of school readiness 1 

g. Child gains on measures of academic achievement 1 

h. Other (SPECIFY) 1 



9. Using the U.S. Department of Education definition of performance indicators as measures that show the degree to 
which key programs achieve desired performance levels, has this program developed performance indicators? 



a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 3 



10. Has your program jointly developed performance indicators with other federal programs? 



a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 



O 

ERIC 



Even Start/ 



(State) 
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11. If your program has developed program performance indicators, how do you and your colleagues in this program 
office use them? (CIRCLE ALL THAT APPLY) 



a. To provide state and local administrators with a common 

language for defining program results 1 

b. To help program staff focus on program goals and objectives 1 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 

f. To provide information about existing or potential problems 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 

h. To provide consumer-oriented information about program performance .... 1 

i. Other (SPECIFY) 1 



12. In response to the requirements of the Government Performance and Results Act, the U.S. Department of Education 
has developed a set of performance indicators for the Even Start Family Literacy (Title I Part B) program. Are 
you aware of the indicators? 

a. Yes 1 

b - No 2 (STOP HERE) 

13. To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability 
of the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 



Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 

districts receiving 
program funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Adult literacy achievement. By fall 2001, 
40 percent of Even Start adults will achieve 
significant learning gains on measures of math 
skills and 30 percent of adults will achieve such 
gains on measures of reading skills. 


1 


2 


3 


4 


5 


1.3: Children’s language development and 
reading readiness. By fall 2001 , 60 percent of 
Even Start children will attain significant gains on 
measures of language development and reading 
readiness. 


1 


2 


3 


4 


5 


1.4: Parenting skills. Increasing percentages of 
parents will show significant gains on measures of 
parenting skills, knowledge and expectations for 
their children. 


1 


2 


3 


4 


5 



Even Start/ (State) 

ERIC 
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14. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state on 
the performance indicators listed in Question 13? 



15. [Where data are collected] If your state collects data on any of the indicators (listed above), how is the information used? 
(CIRCLE ALL THAT APPLY) 

Indicator Indicator Indicator 
U L3 L4 

a. To identify districts and schools for which technical assistance 



is needed 1 2 3 

b. To report to state officials 1 2 3 

c. To report to federal officials 1 2 3 

d. To identify priorities for state-level technical assistance 1 2 3 

e. Other (SPECIFY) 1 2 3 



16. [Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 
(CIRCLE ALL THAT APPLY) [NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) . . 1 



[Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 13, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 13. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4” for 
Not Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 



Could Collect 
from Districts 
and Report 
Fairly Easily 



Possible to Collect 
from Districts & 
Report, But w/ 
Some Difficulty 



Extremely 
Difficult to 

Collect & Not Don’t 
Report Appropriate Know 



a. Indicator 1 . 1 : Adult literacy achievement .. 1 

b. Indicator 1.3: Children’s language 

development & reading readiness 1 

c. Indicator 1.4: Parenting skills 1 



2 



3 4 5 



2 

2 



3 4 5 

3 4 5 



17b. [Where relevant] Why do you consider it inappropriate to collect data on some or all of the indicators listed in Question 
13? 



17c. [Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 13? 



Even Start/ 



(State) 




18 . 



Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 13? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 



Even Start/_ 

coir 



(State) 



5 
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Additional Items for the State Director of Migrant Education 



1. In your opinion, do the state's plans for assessing student mastery of state content standards address 
migratory students' unique needs? 



a. Yes (EXPLAIN) . 1 

b. No (EXPLAIN) . 2 

c. Don't know 3 



2. Approximately how many migrant students are included in your State assessments? (CIRCLE ONE) 



a. None 1 

b. Few 2 

c. Some 3 

d. Many /almost all 4 

e. Don't know 5 



3. Which of the following types of special accommodations are made (if any) to ensure that migrant students 
are included in State assessments? (CIRCLE ALL THAT APPLY) 



a. Coordination between states to share assessment instruments 1 

b. Appointment of state-level personnel to track students moving across state 

and district boundaries with respect to their assessment records 1 

c. School-entry and exit assessments that measure students’ mastery of 
content standards, language proficiency, and grade level or special 

program placement 1 

d. Providing for the inclusion of limited English proficient students 1 

e. Other (specify) . 1 

f. No special accommodations are made 1 



4. To what extent is the curriculum used in migrant summer programs aligned with the standards of 
your state? (CIRCLE ONE) 



a. Not at all 1 

b. A little 2 

c. To some extent 3 

d. To a great extent 4 

e. Don't know 5 



5. To what extent is the curriculum used in migrant summer programs aligned with the standards of 
home-base states? (CIRCLE ONE) 



a. Not at all 1 

b. A little 2 

c. To some extent 3 

d. To a great extent 4 

e. Don't know 5 



DRAFT Migrant/ 




(State) 



1 
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6. How are academic and other records for migrant students who reside in your state being transferred 
when the students move across school district lines within the state? (Please CIRCLE the TWO most 
frequent ways that records are transferred across school district lines). 

a. Forwarded through an electronic database (e.g., a state or multi-state system) . . 1 

b. Given to the student or parent to hand-carry 1 

c. Mailed 

d. Faxed 1 

e. Other (SPECIFY) 1 

Between states? (Please CIRCLE the TWO most frequent ways that records are transferred between states). 

a. Forwarded through an electronic database (e.g., a state or multi-state system) . . 1 

b. Given to the student or parent to hand-carry 1 

c. Mailed 

d. Faxed 1 

e. Other (SPECIFY) 1 

7. What is your State doing to ensure that migrant students benefit from all available and appropriate 
programs and services other than the MEP? 



8. Using the U.S. Department of Education definition of performance indicators as measures that show the degree to 
which key programs achieve desired performance levels, has this program developed performance indicators? 

a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 2 

9. Has your program jointly developed performance indicators with other federal programs? 

a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 



10. If your program has developed program performance indicators, how do you and your colleagues in this program 
office use them? (CIRCLE ALL THAT APPLY) 

a. To provide state and local administrators with a common 

language for defining program results 

b. To help program staff focus on program goals and objectives 

c. To provide program outcome data for administrative planning 

d. To enable this program to engage in cross-program coordination .... 

e. To monitor the progress of local programs or subgrantees 

f. To provide information about existing or potential problems 

g. To keep the state board of education or legislature informed 

about the program’s progress 

h. To provide consumer-oriented information about program performance 

i. Other (SPECIFY) 



1 

1 

1 

1 

1 

1 

1 

1 

1 



O 

ERIC 



DRAFT Migrant/_ 



(State) 
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11. In response to the requirements of the Government Performance and Results Act, the U.S. Department of Education 
has developed a set of performance indicators for the Education of Migratory Children (Title I Part C) program. 
Are you aware of the indicators? 

a. Yes 1 

b. No 2 (STOP HERE) 

12. To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability 
of the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 



Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 
districts 
receiving 
program funds 


Yes, from 
a sample of 
districts 


No 


Don’t 

Know 


Other 


3.1: Inter- and intrastate coordination. SEAs and 
LEAs will demonstrate increased interstate and 
intrastate coordination to improve educational 
continuity for migrant students. Measures of 
coordination include joint products resulting from these 
formal agreements, meetings, or conferences to 
promote coordination; coordinated guidance to 
grantees; and joint planning by local staff from all 
available programs. 


1 


2 


3 


4 


5 


3.4: Program coordination. Federal, SEA, and LEA 
staff working with Title I, Part A and Part C, and other 
federally funded programs, will demonstrate increasing 
levels of substantive collaboration to meet the unmet 
needs of migrant children. 


1 


2 


3 


4 


5 





13. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state on 
the performance indicators listed in Question 12? 



14. [Where data are collected] If your state collects data on any of the indicators listed in Question 12, how is the information 
used? (CIRCLE ALL THAT APPLY) 

Indicator Indicator 

H M 

a. To identify districts and schools for which technical assistance 



is needed 1 2 

b. To report to state officials 1 2 

c. To report to federal officials 1 2 

d. To identify priorities for state-level technical assistance 1 2 

e. Other (SPECIFY) 1 2 



DRAFT Migrant/ (State) 3 
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15. 



[Where data are collected] In which of the following ways, if any, does .your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 

[NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) . . 1 



16a. [Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 12, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 12. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for 
Not Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 







Could Collect 


Possible to Collect 


Extremely . 








from Districts 


from Districts & 


Difficult to 








and Report 


Report, But w / 


Collect & Not 


Don’t 






Fairlv Easilv 


Some Difficultv 


Report Approoriate 


Know 


a. 


Indicator 3.1: Inter- and intrastate 
coordination 


i 


2 


3 4 


5 


b. 


Indicator 3.4: Program coordination . 


i 


2 


3 4 


5 


16b. 


[Where relevant] Why do you consider it inappropriate to 


collect data on some or all of the indicators listed in Question 



12 ? 



16c. [Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 12? 



17. Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 12? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) . . . . . 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 



j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 



DRAFT Migrant/ (State) 4 



Additional Items for the State Director of Title I, Part D, Subpart 2: 
Local Prevention and Intervention Programs for Children and Youth Who 
Are Neglected, Delinquent, or At-Risk of Dropping Out 



Does the SEA award Subpart 2 grants to local education agencies on a competitive basis or 



on the basis of some formula? 

a. Grants are awarded on a competitive basis 1 

b. Grants are awarded based on a formula (SPECIFY) 2 (SKIP to Q4) 



c. Other (SPECIFY) 3 (SKIP to Q4) 



In the past 12 months, how many local subgrant awards did this state make under Subpart 2? 

What was the total number of local applications received for subgrant awards? 

What is the total amount of Subpart 2 funds allocated to local education agencies 
through subgrant awards? 

In 1997-98, what were the three leading selection factors for awarding grants to local education agencies? Among 
the following, CIRCLE the THREE factors which were given the most weight in selecting local education agencies 
for funding. (CIRCLE THREE RESPONSES ONLY) 



a. District’s capacity to provide the services offered 1 

b. District’s prior experience in serving neglected, delinquent, or at-risk youth . . 1 

c. Quality of the proposed project 1 

d. Number or percentage of neglected, delinquent, or at-risk youth 

residing in the district 1 

e. Number or percentage of youth residing in locally operated correctional 

facilities 1 

f. Severity of the unmet needs of neglected, delinquent, or at-risk youth 1 

g. Number and/or quality of local programs serving neglected, delinquent, 

or at-risk youth 1 

h. All applications were funded 1 

i. Other (SPECIFY) 1 



Using the U.S. Department of Education definition of performance indicators as measures that show 
the degree to which key programs achieve desired performance levels, has this program developed 
performance indicators? 



a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 3 



TV 



(State) 



1 
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Has your program jointly developed performance indicators with other federal programs? 

a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 

If your program has developed program performance indicators, how do you and your colleagues 
in this program office use them? (CIRCLE ALL THAT APPLY) 

a. To provide state and local administrators with a common 

language for defining program results 1 

b. To help program staff focus on program goals and objectives 1 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 

f. To provide information about existing or potential problems 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 

h. To provide consumer-oriented information about program performance 1 

i. Other (SPECIFY) __ 1 



In response to the requirements of the Government Performance and Results Act, the U.S. 

Department of Education has developed a set of performance indicators for the Title I Part D 
program. Are you aware of the indicators? 

a. Yes 1 

b. No 2 (STOP HERE) 



2 



10 . 



To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability 
of the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 



Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 

districts receiving 
program funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Academic achievement. The number of N, 
D, and at-risk children and youth who will 
progress toward a high school diploma or GED 
while institutionalized will increase. 


1 


2 


3 


4 


5 


2.1: Institution-wide programs. The number of 
institutions that will operate institution-wide 
programs integrating other federal and state 
programs to improve curriculum and instruction 
across the institution will increase. 


1 


2 


3 


4 


5 


2.2: Innovative transition programs. State and 
local programs will develop innovative strategies 
that help institutionalized students make a 
successful transition from an institution back to 
the community, either to further their education or 
to obtain employment. 


1 


2 


3 


4 


.5 



1 1 . [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state 
on the performance indicators listed in Question 10? 



12 . 



[Where data are collected] If your state collects data on any 
used? (CIRCLE ALL THAT APPLY) 



of the indicators listed in Question 10, how is the information 

Indicator Indicator Indicator 
U 2A 12 



a. 


To identify districts and schools for which technical assistance 
is needed 


1 


2 


b. 


To report to state officials 


1 


2 


c. 


To report to federal officials 


1 


2 


d. 


To identify priorities for state-level technical assistance 


1 


2 


e. 


Other (SPECIFY) 


1 


2 



3 

3 

3 

3 

3 



M or D/ 
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13 . 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 
[NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) . . l 



14a. [Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 10, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 10. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for 
Not Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 



a. 

b. 

c. 





Could Collect 


Possible to Collect 


Extremely 








from Districts 


from Districts & 


Difficult to 








and Report 


Report, But w / 


Collect & 


Not 


Don’t 




Fpirlv Easilv 


Some Difficultv . . 


Report 


Appropriate 


Know 


Indicator 1.1: Academic achievement . 


. . . i 


2 


3 


4 


5 


Indicator 2. 1 : Institution-wide programs 


. . . i 


2 


3 


4 


5 


Indicator 2.2: Innovative transition programs 1 


2 


3 


4 


5 



14b. [Where relevant] Why do you consider it inappropriate to collect data on some or all of the indicators listed in Question 
10 ? 



14c. [Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 10? 



15. Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 10? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 



Nnr D / 

ERIC 



(State) 



4 
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Additional Items for the State Coordinator of the Eisenhower Professional 

Development Program (Title II) 



1. Using the U.S. Department of Education definition of performance indicators as measures that show the degree to 
which key programs achieve desired performance levels, has this program developed performance indicators? 



a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 3 

2. Has your program jointly developed performance indicators with other federal programs? 

a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 



3. If your program has developed program performance indicators, how do you and your colleagues in this program 
office use them? (CIRCLE ALL THAT APPLY) 



a. To provide state and local administrators with a common 

language for defining program results 1 

b. To help program staff focus on program goals and objectives 1 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 

f. To provide information about existing or potential problems 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 

h. To provide consumer-oriented information about program performance 1 

i. Other (SPECIFY) 1 



4. In response to the requirements of the Government Performance and Results Act, the U.S. Department of Education 

has developed a set of performance indicators for the Eisenhower Professional Development Program-Title II. Are 
you aware of the indicators? 

a. Yes 1 

b. No 2 (STOP HERE) 



Title II/ (State) 

ERIC 
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5. To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability 
of the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 





SEA 

Currently Collects Data? 


Indicator 


Yes, from 
ALL 

districts receiving 
program funds 


Yes, 
from a 
sample 
of 

HiQtrirtQ 


No 


Don’t 

Know 


Other 


1.1: Teachers* skills and classroom instruction. 
By 1998, over 50% of a sample of teachers will 
show evidence that participation in 
Eisenhower-assisted professional development has 
resulted in improvement in their knowledge and 
skills, and by 1999 in an improvement in classroom 
instruction. 


1 


2 


3 


4 


5 


3.2: Duration. By 1998, 35% of teachers 
participating in district-level Eisenhower assisted 
activities will participate in activities that are a 
component of professional development that extends 
over the school year; by 2000, over 50% will. 


1 


2 


3 


4 


5 


4. 1 : High-poverty schools. The proportion of 
teachers participating in Eisenhower-assisted 
activities who teach in high-poverty schools will 
exceed the proportion of the national teacher pool 
who teach in high-poverty schools. 


1 


2 


3 


4 


5 



6. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state 
on the performance indicators listed in Question 5? 



7. 



[Where data are collected] If your state collects data on any of the indicators listed in Question 5, how is the information 
used? (CIRCLE ALL THAT APPLY) 

Indicator Indicator Indicator 



LI 

a. To identify districts and schools for which technical assistance 



is needed 1 

b. To report to state officials 1 

c. To report to federal officials 1 

d. To identify priorities for state-level technical assistance 1 

e. Other (SPECIFY) 1 



12 4 A 



2 

2 

2 

2 

2 



3 

3 

3 

3 

3 




Title II/ 
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8. [Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 
(CIRCLE ALL THAT APPLY) [NOTE: ANSWER THIS QUESTION AND STOP] 

a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) • • 1 



[Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 5, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 5. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for Not 
Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 







Could Collect 


Possible to Collect 


Extremely 










from Districts 


from Districts & 


Difficult to 










and Report 


Report, But w/ 


Collect & 


Not 


Don’t 






Fairlv Easilv 


Some Difficulty 


Report 


Appropriate 


Know 


a. 


Indicator 1.1: Teachers’ skills and 
classroom instruction 


. . . . 1 


2 


3 


4 


5 


b. 


Indicator 3.2: Duration 


i 


2 


3 


4 


5 


c. 


Indicator 4.1: High-poverty schools . . 


i 


2 


3 


4 


5 


9b. 


[Where relevant] Why do you consider it inappropriate to 


collect data on some or 


all of the indicators listed in 


Question 



5? 



9c. [Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 5? 



10. Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 5? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 
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Additional Items for the State Director of the 
Technology Literacy Challenge Fund (TLCF) 



1. How many local subgrant awards did this state make for fiscal year 1998? 

2. What was the total number of local applications received for subgrant awards in fiscal year 1998? 

2a. What was the total amount of funds requested in the 1997-98 subgrant applications? 

3. In 1997-98, what criteria did your state use to allocate local grants to high need and high poverty school districts? 
(CIRCLE ALL THAT APPLY) 

a. District’s capacity to provide supporting resources (e.g., services, software 
and print resources) to ensure successful and effective use of technologies 

acquired under the program 1 

b. Quality of district’s plans to coordinate the technology provided under TLCF 

with other grant funds available for technology purchases 1 

c. Extent to which other funds are available for technology purchases and use . . . 1 

d. Appropriateness of the proposed technology purchases 1 

e. Extent to which district plan is of sufficient scope to improve student learning . 1 

f. Extent to which district has developed a plan for integrating technologies 

acquired under TLCF into the school curriculum 1 

g. Extent to which district will involve parents, public libraries, business and 

community leaders in the development of an education technology plan 1 

h. Other (SPECIFY) 1 

4. What evidence, if any, does the state have that educational technology is associated with changes (either increases 
or decreases) in student performance in the core curriculum (e.g., reading/language arts, mathematics, science, 
social studies)? (CIRCLE ALL THAT APPLY) 



a. Student performance data that links to information on 

classroom or school resources (SPECIFY Type and Source) 1 



b. Teacher survey data (Collected by state/district/school? ) 1 

c. Student survey data (Collected by state/district/school? ) 1 

d. District survey data (Collected by state? ) 1 

e. School site visits (Conducted by state/district? ) 1 

f. Analyses of requests for information/ assistance from the field 1 

g. Website “hits” 1 

h. Informal conversations with local project staff 1 

i. Other (SPECIFY) 1 

j. None 1 



Technology Literacy 
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What evidence, if any, does the state have that educational technology is actually being used for instruction'- 1 
(CIRCLE ALL THAT APPLY) 



a. Teacher survey data (Collected by state/district/school? ) 1 

b. Student survey data (Collected by state/district/school? ) l 

c. District survey data (Collected by state? ) i 

d. School site visits (Conducted by state/district? ) l 

e. Analyses of requests for information/assistance from the field 1 

f. Website “hits” 1 

g. Informal conversations with local project staff 1 

h. Other (SPECIFY) j 

i. None i 



Does the state monitor access to educational technology for: (CIRCLE ONE RESPONSE FOR EACH ROW) 

Yes No 



a. Students with disabilities \ 2 

b. Students with limited English proficiency 1 2 

c. Students in remote rural schools \ 2 



d. High poverty schools (i.e. , 75 % or more students eligible for 

free/reduced-price school lunch) 

e. Other (SPECIFY) 



Has the state used its TLCF funds to support professional development in educational technology? 

a. Yes (SPECIFY percent of TLCF funds used) 1 

b. No 9 



In the past three years, has the state’s emphasis on professional development in educational technology: 

a. Increased j 

b. Decreased 2 

c. Remained about the same -j 



Are teacher preparation programs at state colleges and universities required to provide training in the use of educational 
technology as a condition for receiving state accreditation? 

a. Yes 1 

b. No 0 



Is training in technology-related standards or competencies required for teachers in your state? 

a. Yes 1 

b. No o 



Technology Literacy 
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1 1 . Does your state assess student proficiency in the use of technology? 



l 

No ’ 2 (SKIP to Q13a) 



12. What types of student proficiencies in the use of technology are assessed and at what grade levels? 



13a. Has your state provided technical assistance to districts that are preparing educational technology plans in 
order to qualify for the Federal Communication Commission’s Universal Services education rate (E-rate)? 

2 Yes 1 

b . No 2 (SKIP to Q14) 



13b. Were the technology plans reviewed by an independent panel? 

a. Yes 

b. No 



13c. On what basis were districts selected to receive technical assistance in preparing their educational technology plans? 
(CIRCLE ALL THAT APPLY) 

a. Districts that request help 1 _ 

b. High-poverty districts 1 _ 

c. Districts with minimal technological capacity and/or expertise 1 _ 

d. Other (SPECIFY) 1 



14. In your estimation, to what extent has your program been able to meet local demand for technical assistance in the 
following areas? (CIRCLE ONE RESPONSE FOR EACH ROW) 



Great Some A Not at 

Extent Extent Little All NA 



a. Preparing a educational technology plan 1 

b. Preparing an application for the E-rate 1 

c. Preparing teachers to use educational technology 

in the classroom 1 

d. Other (SPECIFY) 1 



2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 



15. Does your state provide districts and/or schools with information about effective models for using 
educational technology? 



a. Yes (SPECIFY school, district, or both) 1 

b. No 2 
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In your state, what among the following are barriers to using technology for instructional purposes? 
(CIRCLE ONE RESPONSE FOR EACH ROW) 

Major Minor Not a 

Barrier Barrier Barrier 


Don’t 

Know 


a. Insufficient telephone lines in schools 


1 


2 


3 


4 


b. Insufficient equipment (e.g., number of computers) . 


1 


2 


3 


4 


c. Lack of technical support or advice 


1 


2 


3 


4 


d. Lack of or inadequately trained staff 

e. Lack of teacher awareness regarding ways to 


1 


2 


3 


4 


integrate technology into the curriculum 

f. Lack of software that is integrated with 


1 


2 


3 


4 


the school curriculum 

g. Lack of software that is appropriate for special 

populations (e.g., students with disabilities. 


1 


2 


3 


4 


students with limited English proficiency, etc.) .... 


1 


2 


3 


4 


h. Other (SPECIFY) 


1 


2 


3 


4 



Using the U.S. Department of Education definition of performance indicators as measures that show the degree to which 
key programs achieve desired performance levels, has this program developed performance indicators? 



a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 3 

18 . Has your program jointly developed performance indicators with other federal programs? 

a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 



If your program has developed program performance indicators, how do you and your colleagues in this program office 
use them? (CIRCLE ALL THAT APPLY) 



a. To provide state and local administrators with a common 

language for defining program results 1 

b. To help program staff focus on program goals and objectives 1 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 

f. To provide information about existing or potential problems . 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 

h. To provide consumer-oriented information about program performance 1 

i. Other (SPECIFY) 1 
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20. In response to the requirements of the Government Performance and Results Act, the U.S. Department of 
Education has developed a set of performance indicators for the Technology Literacy Challenge Fund 
program. Are you aware of the indicators? 



a. Yes 

b. No 



1 

2 (STOP HERE) 



21. To assist in updating and improving the performance indicators, we are attempting to gauge the extent to 
which the following information is currently collected by your state. Please provide your best estimates 
as to the availability of the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 



Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 

districts receiving 
program funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


3.3: Staff training. Increasing proportions 
of practicing and prospective teachers, school 
administrators, and school librarians will 
receive professional development that enables 
them to effectively use education technology to 
help students learn. 


1 


2 


3 


4 


5 


4.2: Access in high-poverty schools. The 
access to education technology in high-poverty 
schools will be comparable to that in other 
schools. 


1 


2 


3 


4 


5 



22. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state 
the performance indicators listed in Question 21? 



23. 



[Where data are collected] If your state collects data on any of the indicators listed in Question 21, how is the 
information used? (CIRCLE ALL THAT APPLY) 

Indicator Indicator 
3 3 42 



a. 


To identify districts and schools for which technical assistance 

ie DPpHpH ..... 


1 


2 


b. 


To report to state officials 


1 


2 


c. 


To report to federal officials 


1 


2 


d. 


To identify priorities for state-level technical assistance 


1 


2 


e. 


Other (SPECIFY! 


1 


2 
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24 . 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required’ 
(CIRCLE ALL THAT APPLY) [NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) 



[Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 21, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 21. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for 
Not Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 



Could Collect 
from Districts 
and Report 
Fairly Easily 



a. Indicator 3.3: Staff training 1 

b. Indicator 4.2: Access in high- 

poverty schools 1 



Possible to Collect 


Extremely 






from Districts & 


Difficult to 






Report, But w / 


Collect & 


Not 


Don’t 


Some Difficulty . . 


Report 


Appropriate 


Know 


2 


3 


4 


5 


2 


3 


4 


5 



25b. 



[Where relevant] Why do you consider it inappropriate to collect data on some or all of the indicators listed in Question 



[Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 21? 



Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 21? (CIRCLE TH REE ) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) ' " j 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 
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Additional Items for the State Coordinator of 
the Safe and Drug-Free Schools and Communities Program 



From which of the following sources are data used to determine which districts have the greatest need for these 
funds? (CIRCLE ALL THAT APPLY) 

a. Applicant districts 1 

b. Our program office: the state Safe and Drug-Free Schools Program collects 

some/most/all of the district-level data needed (SPECIFY) 1 

c. Other SEA program offices 1 

d. Other state agencies 1 

e. Other (SPECIFY) 1 

In your estimation, what are the benefits, if any, to concentrating 30 percent of available program funds in districts of 
greatest need? 



In your opinion, what are the drawbacks, if any, to concentrating 30 percent of available program funds in districts of 
greatest need? 



Using the U.S. Department of Education definition of performance indicators as measures that show the degree to 
which key programs achieve desired performance levels, has this program developed performance indicators? 

a. Yes (OBTAIN A COPY) •1 

b. No 2 

c. Developing 3 — 

Has your program jointly developed performance indicators with other federal programs? 

a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 — 

If your program has developed program performance indicators, how do you and your colleagues in this program 
office use them? (CIRCLE ALL THAT APPLY) 

a. To provide state and local administrators with a common 

language for defining program results 1 — 

b. To help program staff focus on program goals and objectives 1 _ 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 _ 

f. To provide information about existing or potential problems 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 _ 



h. To provide consumer-oriented information about program performance 1 

i. Other (SPECIFY) 1 
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7. In response to the requirements of the Government Performance and Results Act, the U.S. Department of Education 
has developed a set of performance indicators for the Safe and Drug-Free Schools Program (Title IV). Are you 
aware of the indicators? 

a. Yes 1 

b. No 2 (STOP HERE) 

8. To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability of 
the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 





SEA 

Currently Collects Data? 


Indicator 


Yes, from 
ALL 
districts 
receiving 
program funds 


Yes, from 
a sample 
of districts 


No 


Don’t 

Know 


Other 


! 

6.1: Gun-Free Schools Act notifications and expulsions. 

By 1998, all liEAs receiving ESEA funds will have policies 
requiring the expulsion of students who bring firearms to 
school and requiring notification of law enforcement. 


1 


’ 2 


3 


4 


5 


8.3: Approval of LEA applications. All states will use 
performance indicators to make decisions regarding 
approval of LEA applications for funding. [LEAs are 
required to have performance indicators (called 
"measurable goals & objectives" in the statute) in their 
applications/plans] 


1 


2 


3 


4 


5 



9. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state on 
the performance indicators listed in Question 8? 



10 . 



[Where data are collected] If your state collects data on any of the indicators listed in Question 8, how is the information 
used? (CIRCLE ALL THAT APPLY) 



a. 

b. 

c. 

d. 

e. 



To identify districts and schools for which technical assistance 

is needed 

To report to state officials 

To report to federal officials 

To identify priorities for state-level technical assistance 

Other (SPECIFY) 



Indicator Indicator 
6J, 83 



1 

1 

1 

1 

1 



2 

2 

2 

2 

2 
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11. 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 
(CIRCLE ALL THAT APPLY) [NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) . . 1 



[Where data are NOT collected] If you are not collecting data on any or all of the indicators listed in Question 8, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 8. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for Not 



Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 

Could Collect 
from Districts 
and Report 
Fairly Easily 

a. Indicator 6.1: Gun-Free Schools Act 

notifications and expulsions 1 

b. Indicator 8.3: Approval of LEA applications 1 



Possible to Collect 


Extremely 






from Districts & 


Difficult to 






Report, But w/ 


Collect & 


Not 


Don’t 


Some Difficultv 


Report 


AonroDriate 


Know 


2 


3 


4 


5 


2 


3 


4 


5 



12b. [Where relevant] Why do you consider it inappropriate to collect data on some or all of the indicators listed in Question 
8 ? 



12c. [Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 8? 



13. Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 8? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCS SO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 



Title IV/ 



(State) 
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Additional Items for State Title VI Coordinator 



1 . In adjusting your formula for children whose education imposes a higher than average cost per child, which of the 
following factors do you use? (CIRCLE ALL THAT APPLY) 

a. Children living in areas with high concentrations of low-income 



families 1 

b. Children from low-income families 1 

c. Sparsely populated areas 1 

d. Other (SPECIFY) . 1 



e. No adjustment made 1 



2. What data source(s) do you use when you adjust your formula? 



3. Using the U.S. Department of Education definition of performance indicators as measures that show the degree to which 
key programs achieve desired performance levels, has this program developed performance indicators? 



a. Yes (OBTAIN A COPY) 1 

b. No 2 

c. Developing 3 

4. Has your program jointly developed performance indicators with other federal programs? 

a. Yes (SPECIFY PROGRAMS) 1 

b. No 2 



5. If your program has developed program performance indicators, how do you and your colleagues in this program office 
use them? (CIRCLE ALL THAT APPLY) 



a. To provide state and local administrators with a common 

language for defining program results 1 

b. To help program staff focus on program goals and objectives 1 

c. To provide program outcome data for administrative planning 1 

d. To enable this program to engage in cross-program coordination 1 

e. To monitor the progress of local programs or subgrantees 1 

f. To provide information about existing or potential problems 1 

g. To keep the state board of education or legislature informed 

about the program’s progress 1 

h. To provide consumer-oriented information about program performance .... 1 

i. Other (SPECIFY) 1 
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6. In response to the requirements of the Government Performance and Results Act, the U.S. Department of 

Education has developed a set of performance indicators for the Title VI (Innovative Education Program Strategies) 
program. Are you aware of the indicators? 

a. Yes 1 

b- No 2 (STOP HERE) 



To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which 
the following information is currently collected by your state. Please provide your best estimates as to the 
availability of the following information. 



Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 

districts receiving 
program funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


1.1: Reform efforts. The use of Title VI funds 
will show evidence that the activities supported 
are integral to achieving district reform plans. 


1 


2 


3 


4 


5 



8. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state on 
the performance indicator listed in Question 7? 



9. [Where data are collected] If your state collects data on the indicator listed in Question 7, how is the information used? 
(CIRCLE ALL THAT APPLY) 

Indicator 

II 

a. To identify districts and schools for which technical assistance 



is needed 1 

b. To report to state officials 1 

c. To report to federal officials 1 

d. To identify priorities for state-level technical assistance 1 

e. Other (SPECIFY) 1 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicator for which data have been requested and/or required? 
(CIRCLE ALL THAT APPLY) [NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) ” ’ ’ ’ i 
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(State) 



2 




1 la. [Where data are NOT collected] If you are not collecting data on the indicator listed in Question 7, how difficult would it be to 
start? Please indicate the extent to which data are available from districts for the indicator listed in Question 7 [NOTE: If you do 
not believe it would be appropriate to collect data on the indicator, circle 4" for Not Appropriate ) 



a. 



Indicator 1.1: Reform efforts 



Could Collect 
from Districts 
and Report 
Fairly Easily 



Possible to Collect 
from Districts & 
Report, But w / 
Some Difficulty 



Extremely 
Difficult to 

Collect & Not Don’t 
Report Appropriate Know 



1 2 3 4 5 



lib. [Where relevant] Why do you consider it inappropriate to collect data the indicator listed in Question 7? 



1 lc. [Where relevant] Why would it be extremely difficult to collect and report data on the indicator listed in Question 7? 



12. Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on the indicator listed in Question 7? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 
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Additional Items for the State Goals 2000 Coordinator 



1. On what basis do districts award the 50 percent of funds that are to be made available to schools “with a 
special need for such assistance”? (CIRCLE ONE) 



a. Primarily poverty 1 

b. Primarily low achievement 2 

c. Primarily another basis (SPECIFY) 3 



d. Don’t know 4 



2. What factors are considered and what targeting methods used in making local reform subgrant awards? 



Factors Considered in 
Making 

Subgrant Awards 


Priority Given Each 
Factor [RANK, in 
order of priority, all 
that apply (e.g., 1, 2, 
3, etc.)] 


Primary Targeting Method (select ONE of the 
following): 

1: Weight applied during application review for 
purposes of making a funding decisions 
2: Weight applied during application review for 
purposes of determining subgrant size 
3: Targeting technical assistance to high-need 
LEAs and consortia of LEAs to assist them in 
preparing their applications 
4: Pre-grant awards to high-need LEAs and 
consortia of LEAs to assist them in preparing 
their applications 


a. Purpose or type of need being addressed (e.g. , 
standards and/or assessment development) 






b. Consortia of LEAs 






c. LEAs/consortia with high percentages of low- 
income students 






d. LEAs/consortia with high percentages of low- 
achieving students 






e. Maintenance of rural/urban or regional 
balance among subgrantees 






f. Size or scope of project 






g. LEAs/consortia with community partnerships 






h. Quality of application/local plan 






i. Other (SPECIFY) 
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3 . 



To what extent do each of the following change from year to year with respect to local reform subgrant 
awards? (CIRCLE ONE RESPONSE FOR EACH ROW) 

Great Some A Not at 
Extent Extent Little All 



a. Factors considered in making subgrant awards .... 1 

b. Priority assigned each factor considered in 

making subgrant awards 1 

c. Targeting method 1 



2 3 4 

2 3 4 

2 3 4 



4. What factors are considered and what targeting methods used in making preservice and professional development 
subgrant awards? 



Factors Considered in 
Making 

Subgrant Awards 


Priority Given Each 
Factor [RANK, in 
order of priority, all 
that apply (e.g., 1, 2, 
3, etc.)] 


Primary Targeting Method (select ONE of the 
following): 

1: Weight applied during application review for 
purposes of making a funding decisions 
2: Weight applied during application review for 
purposes of determining subgrant size 
3: Targeting technical assistance to high-need 
LEAs and consortia of LEAs to assist them in 
preparing their applications 
4: Pre-grant awards to high-need LEAs and 
consortia of LEAs to assist them in preparing 
their applications 


a. Purpose or type of need being addressed (e.g. , 
standards and/or assessment development) 






b. Consortia of LEAs 






c. LE As/consortia with high percentages of low- 
income students 






d. LE As/consortia with high percentages of low- 
achieving students 






e. Maintenance of rural/urban or regional 
balance among subgrantees 






f. Size or scope of project 






g. LEAs/consortia with community partnerships 






h. Quality of application/local plan 






i. Other (SPECIFY) 
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To what extent do each of the following change from year to year with respect to preservice and professional 
development subgrant awards? (CIRCLE ONE RESPONSE FOR EACH ROW) 



6 . 



Great Some A Not at 
Extent Extent Little All 



a. Factors considered in making subgrant awards .... 1 

b. Priority assigned each factor considered in 

making subgrant awards 1 

c. Targeting method 1 



2 3 4 

2 3 4 

2 3 4 



In response to the requirements of the Government Performance and Results Act, the U.S. Department of Education 
has developed a set of performance indicators for the Goals 2000 program. Are you aware of the indicators? 

a. Yes 1 

b. No . . 2 (STOP HERE) _ 



7. To assist in updating and improving the performance indicators, we are attempting to gauge the extent to which the 
following information is currently collected by your state. Please provide your best estimates as to the availability of 
the following information. (CIRCLE ONE RESPONSE FOR EACH INDICATOR) 



Indicator 


SEA 

Currently Collects Data? 


Yes, from 
ALL 

districts receiving 
program funds 


Yes, from a 
sample of 
districts 


No 


Don’t 

Know 


Other 


2.1: Participation in reform efforts. By 1999, as 
many as half of the state’s school districts will actively 
participate in standards-based reform. 


1 


2 


3 


4 


5 


4.2: Schools alignment of key processes. Surveys of 
principals and teachers in states with standards will 
indicate that schools have aligned curriculum, 
instruction, professional development and assessment to 
meet challenging state or local standards. 


1 


2 


3 


4 


5 



8. [Where data are collected] What kinds of problems, if any, do you experience in collecting information in your state on 
the performance indicators listed in Question 7? 
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[Where: data are collected] If your state collects data on any of the indicators listed in Question 7, how is the information 
used? (CIRCLE ALL THAT APPLY) 

Indicator Indicator 
2A 12 

a. To identify districts and schools for which technical assistance 



is needed 1 2 

b. To report to state officials 1 2 

c. To report to federal officials 1 2 

d. To identify priorities for state-level technical assistance 1 2 

e. Other (SPECIFY) 1 2 



[Where data are collected] In which of the following ways, if any, does your state initiate corrective action against local 
school districts that fail to submit data to the state on the indicators for which data have been requested and/or required? 
(CIRCLE ALL THAT APPLY) [NOTE: ANSWER THIS QUESTION AND STOP] 



a. It does not 1 

b. Assist local school districts in developing a plan to collect the data 1 

c. Withhold program funds 1 

d. Other (SPECIFY) 



[Where data are NOT collected] If you are not collecting data on any or all cf the indicators listed in Question 7, how difficult 
would it be to start? Please indicate the extent to which data are available from districts for each of the indicators listed in 
Question 7. [NOTE: If you do not believe it would be appropriate to collect data on any or all of the indicators, circle 4" for Not 
Appropriate ) (CIRCLE ONE RESPONSE FOR EACH ROW) 







Could Collect 


Possible to Collect 


Extremely 










from Districts 


from Districts & 


Difficult to 










and Report 


Report, But w/ 


Collect & 


Not 


Don’t 






Fairlv Easilv 


Some Difficulty 


Report 


Appropriate 


Know 


a. 


Indicator 2.1: Participation in reform 
efforts 


i 


2 


3 


4 


5 


b. 


Indicator 4.2: Schools alignment of 
key processes 


i 


2 


3 


4 


5 



[Where relevant] Why do you consider it inappropriate to collect data on some or all of the indicators listed in Question 
7? 



[Where relevant] Why would it be extremely difficult to collect and report data on some or all of the indicators listed in 
Question 7? 



Goals 2000/ 



[C 



(State) 



4 



223 



12 . 



Which THREE of the following agencies or organizations would you most likely contact to get assistance with respect to 
collecting data on any of the indicators listed in Question 7? (CIRCLE THREE) 



a. U.S. Department of Education program contact 1 

b. U.S. Department of Education Regional Service Team 1 

c. Regional Educational Laboratories (SPECIFY NAME ) 1 

d. Comprehensive Centers (SPECIFY NAME ) . . . 1 

e. Professional associations (e.g., CCSSO) 1 

f. Institutions of higher education 1 

g. Other states 1 

h. Private non-governmental organizations or foundations 1 

i. Other (SPECIFY) 1 

j. I wouldn’t; I don’t believe it is appropriate to collect such data 1 
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Additional Items for the State Coordinator of the 
Education Flexibility Partnership Demonstration Program or “Ed Flex” 



[NOTE: The following set of questions (F1-F8) are applicable only to program administrators in states that are 
participating in the Education Flexibility Partnership Demonstration Program or Ed-Flex under the Goals 2000: 
Educate America Act] 

FI. To what extent do you find that Ed-Flex authority has given your state more administrative flexibility than it 
had before your state was granted Ed-Flex status? (CIRCLE ONE) 



a. To a considerable extent 1 

b. Somewhat 2 

c. Not at all 3 (SKIP to F3) 

d. Don’t know 4 (SKIP to F3) 



F2. Please elaborate on the ways in which your state’s flexibility has increased. How do you know this? 



F3. How would you characterize the statewide use of the Ed-Flex waiver authority in your state? (CIRCLE ONE) 



a. The states is underutilizing its capacity 1 

b. The state is overutilizing its capacity 2 

c. The state is using its Ed-Flex waiver authority at a reasonable level 3 

d. Don’t know 4 



F4. How would you characterize district-level use of the Ed-Flex waiver authority? (CIRCLE ONE) 



a. The districts are underutilizing this opportunity 1 

b. The districts are overutilizing this opportunity 2 

c. The districts are using the Ed-Flex waiver authority at a reasonable level 3 

d. Don’t know 4 



F5a. Since your state received Ed-Flex status, how many districts in your state 
have received waivers? 

F5b. How many districts have lost their waiver authority as a result of 
poor student performance? 

F5c. [If respondent answered “0" to F5b] Does your state revoke district waivers due to poor student performance? 



a. Yes 1 

b. No 2 

c. Don’t know 3 
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F5d. What criteria, if any, does your state use for holding districts accountable for their waiver authority? 



F6. Which of the following best describes the monitoring practices in your state regarding districts 
that have received waivers? (CIRCLE ONE) 



a. Districts with waivers are monitored more frequently 1 

b. Districts with waivers are monitored less less frequently 2 

c. Districts with waivers are monitored about as often as any other district 3 

d. Other (SPECIFY) 4 



F7. To what extent, if at all, did the Ed-Flex waiver authority further state reform efforts? 



a. To a great extent 1 

b. To some extent 2 

c. A little 3 

d. Not at all 4 

e. Don’t know; too early to tell 5 



Please explain. 



F8. To what extent, if at all, did the Ed-Flex waiver authority improve the coordination of federal resources with state reform 
efforts? 



a. To a great extent 1 

b. To some extent 2 

c. A little 3 

d. Not at all 4 

e. Don’t know; too early to tell 5 



Please explain. 
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